Where's all my Data Twitter? Coming Soon says Twitter CEO

Day one of the Web 2.0 Summit, San Francisco, was concluded with a Grand Ballroom dinner and an interview with Dick Costolo, CEO of Twitter.com (@dickc), which for me, was the most insightful and candid interview of the day

The interview ended with one of the toughest questions of the night. Dick Costolo commented to host John Battelle: "You had to invite just one more question!".

The question came from Washington based Alex Howard (@digiphile) of O'Reilly's Radar Site (http://radar.oreilly.com/alexh): "When are we going to get our data?".

Alex stated that although he has tweeted more than 49,000 times but only a few hundreds are available to him. (ed: Well done Alex for raising an important question on behalf of the whole Twittersphere and it is worth noting that Google has a dedicated project called Data Liberation Front that tackles that issue)

Dick went on to explain that since he stated working with Twitter just over 2 years ago, the key focus for their engineering team was on stopping the #FailWhale downtime.

Recently a massive migration project to move Twitter's infrastructure was completed in order to avoid this downtime and to enable them to keep scaling.

To put some numbers on the scale of Twitter today: Twitter now manages over 250 million tweets per day or more than a billion tweets in less than a week!.

Logins have risen from 30 to 50% and mobile usage is growing 40% quarter on quarter. In addition, the recent integration into Apple's iOS5, has made tweeting from your iPhone, iTouch and iPad frictionless and Twitter have already seen a huge uplift in activity as a result.

Dick did not share any numbers on this, but did say that the results were "Better than they hoped!". Dick finally answered the question by saying "Twitter have a finite number of search engineers", but now the infrastructure migration was complete, he said "Discovery is being worked on".

So, at some point in time, it seems that we will all get our hands back on our historic tweets, but only Twitter know when.

On Google+ and Facebook Dick said that Twitter and Apple try to edit things down and make things easy, whilst Google and Facebook are focused on adding functionality and that the Integration of Twitter data back into Google search is still under negotiation.

Also, Twitter noise is apparently being dealt with too. Dick said they've been experimenting with several solutions, many of which have not worked, but now they think they have an answer.

At the same time he stated that Twitter needs to bring more to the surface to get people more engaged and show the users not tell them. Does this mean that Twitter will one day have its own aggregated content portal? Possibly

Finally on the issue of an IPO, Dick was firm on his approach: "We have more money than we are going to need for a long time. We are not looking for a window. But will go public when the company is ready."

On a side note, John Battelle and Dick Costolo are clearly both good friends and business colleagues. When Dick moved to San Francisco two years ago, he moved into the same district which seems to have sealed that relationship. This made the interview highly entertaining and delivered a candid interview which also addressed a number of other interesting issues.

(Ed : the video will be added at a later stage when it goes live on the Web 2.0 Summit 2011 website).