According to thousands of independent scientific tests by RootMetrics across the capital, Vodafone has the edge on O2, recording download speeds of 9.5 Mbps and upload speeds of 4.7 Mbps (upload) compared to O2’s 7.1Mbps (download) and 3.7 Mbps (upload).
Bill Moore CEO and President of Rootmetrics joins us to discuss the findings in more detail.
For more related podcasts click here.
To subscribe in iTunes to receive new podcast episodes for free click here.
Bill, lets start then with an overview of what Rootmetrics do and what your mission is?
First and foremost we are the voice of the consumer. For us, it is about consumer advocacy and taking a look at the major operators in London from a consumer perspective. We take ‘off the shelf’ Smartphone’s that we buy at the retail stores just like everybody else and we test the networks from that means, so we don’t alter these phones, we go indoor and outdoor and run a series of tests across all mobile internet and text and then we score each of the networks based on the speed and reliability of them across, calls, text and then an overall measure.
In the first few months of the service being available, you’ve been testing 4G coverage in and around the city of London. Give us the background behind this study and why it’s been important to test these services so early on in their roll-out.
We have been in the UK for the last couple of years testing the networks inside the initial rollouts and so we have been into London a number of times. This most recent study we did in excess of 86,000 tests and picked more than 100 indoor locations and drove 2,500 miles. When we talk about London it is more than just Zone 1, it is actually all the way out to the smaller towns, to the east it would be out to Southend-by-the sea, north to Saffron Waldon and Maidenhead to the west. So it is a very large and extensive area where we do a random selection and we go and run tests using phones just like a consumer would do. The reason we are doing this is to allow consumers to have better informed choices rather than just listening to marketing messages. We look at what 4G means and how extensive it is so we can go and talk about the speed variances across providers, the reliability of getting on that network and even how widespread the technology actually exists. A lot of advertising suggests that it is everywhere when in fact it is not everywhere, so that is why we do this and do a benchmark if you will that put everyone on a ‘apples with apples’ comparison so there is no bias across the network.
Well you mentioned there that you use devices in the spec that they would be available to consumers, but what is the technology you use behind the scenes to carry out these tests and ensure accuracy?
We take the off the shelf phones and in fact we take a number from each of the operators and benchmark them separately to determine what is going to be the best-case scenario. If you are making a decision as a consumer you might be looking at the latest and greatest phones. We put each operator in best light possible but we do not alter these phones because we want it to be exactly what you and I experience. So our software runs on the top of that and helps automate the tests and we do things to make sure that everybody is tested at the same place at the same time so everything is time synced if you will so it is an absolute variable controlled environment so that we have a direct comparison.
Your study focused on 4G connection speeds in and around the city of London, what were the headline findings that you found?
We take what is called the large urban zone, which is defined by euro stats, which goes out into the suburbs. The important thing to realise is that we take 4G enabled phones where possible in three of the networks but we allow those phones to switch to whatever is available so, if 4G is there great if it is not it might go to 3G or even 2G so it is just like what you and I would experience. Then we take the average of all those tests and report on them in that speed.
What we have found when we look at the mobile Internet is that it is good news for consumers in that everybody is getting faster. O2 and Vodafone are brand new into rolling out 4G and that has occurred over the last 2 months, EE has had a network over the last year and so they have had a bit of a head start but they have also seen improvements. Coming soon is Three, they have not quite rolled out 4G yet but that is coming before the end of the year.
The good news is that the cities are great starts with these rollouts and Vodafone and O2 have seen a doubling of their speeds. In fact Vodafone had a bit of an edge over O2 if we looked at the maximum speed of download about a 9.5 Mbps versus 7.1 Mbps with O2. If we compare and contrast that to EE who have been in the market place a little bit longer and rolled that technology out sooner they have also just rolled out with double speed and that is even a faster technology than 4G. As a result of that we saw them from the last time increase their speeds by 50% and they are actually at about 19.3 Mbps download speed so somewhere between 2 -3 times faster than what we are seeing on O2 and Vodafone.
The important thing to remember here is that, it is not that each test is that much faster, it is an average in that area and they actually have as a result of being out there in the market place sooner and they have a much wider spread percentage of LTE in the overall market place. They vary really in the core city with about 84% of 4G coverage to 64% as we get to the outline areas. We can compare and contrast that to O2 and Vodafone they are more in the region of 45 – 65% in the core city and a little bit less as you get outside where you get only 5% or 18%. What we see typically is these networks come in and generally start in the core centers of towns and to hit the most people as possible to start with and then they build up the masts and technologies as they start to move out. So we are still early and these deployments take more than 60 days, which is really where O2 and Vodafone are at so we would anticipate that they are only going to get faster.
O2 and Vodafone doubled their connection speeds then, tell us a bit more about why one operator can make so much ground on another is it down to the way they deliver services or more about the infrastructure they have access to?
There are a ton of variables here and some of it is related to what allocations they have and the technology choices they made in terms of deploying that. Furthermore, how far they have actually deployed it. In this case it looks like Vodafone is a little more widespread at the moment in terms of deploying 4G than O2. But it’s important to remember that this roll out is still very early on and certainly EE has had much longer to work on deployment.
As important as speed is, we also need to look at reliability as well. So can we get on the network and do what we need to do without interruption like make calls, download a big files, enjoy web and app usage and generally use 4G without problems. What we are seeing on reliability is that although Vodafone are faster on average than O2, O2 are actually more reliable on average. We believe that is a very important factor because if you can’t get on and stay on, speed doesn’t really matter. So that reliability quotient has edged O2 over Vodafone and Three. EE actually has the highest reliability when it comes to mobile internet and part of that is having one technology spread a little wider and not having to switch between the various different flavours of mobile internet technology.
What is important to note is that when we talk about these maximum speeds, it’s like comparing a car with just the one measure of top speed. We wouldn’t buy a car on that one measure alone. So the other components we look at as well include using everyday tasks like web, app usage and emails. Even Three, who don’t have 4G in the marketplace for example, where Vodafone are at 9.5, O2 at 7.1, Three is actually at 5.5 with just pure 3G. So they have one of the fastest underlying 3G networks out there and what is interesting is that while EE have the fastest speed overall they are almost on a par with just 3G alone in terms of being able to get out and do your basic facebook checking and surfing the web. Sometimes maximum speed means a lot and sometimes it may not translate into an everyday task just like a simple App or web use.
There seems to be a lot of work needed outside of the core of London to improve reliability from the data in your study?
As I was mentioning earlier the core of London is where the stronger levels of deployment are occurring and we see the fastest speeds there and the best reliability. In fact as you move out to St. Albans or Maidstone in those types of areas you are going to probably see as much as a 50% drop in speed. If you are in the core of London and you are commuting into London you are absolutely going to want in my opinion the 4G technologies because you can enjoy speeds that are 2 – 4 times anything else that you could enjoy. But if you are in these outline areas and you are not travelling in you may want to delay. You have got to be careful of where that deployment is because we are still early in that process so if you are an early adopter go and get the phones now because it is coming and everybody is building this up quite rapidly. Again, it is a good day for the consumer because this competitive race has only just started heating up.
If you were advising operators on how to improve their services, what would you advise them to do to improve the infrastructure over the next 12 to 24 months?
I think in the large part they are doing what they need to do and we are seeing this deployment of 4G in London and other major cities in the UK actually coming out very strong. The technology is rolling out quickly and even more quickly than we have seen in other parts of the world. We measure everything in the US and the speeds that Londoners are going to be enjoying as we speak is rivaling and even exceeding what we see in for example, New York. The speed aspect is really really good . One of the things we would say though and where we place a lot of importance is reliability, the ability to get on and make my call and do the tasks that I want the mobile and internet be able to do, like send and receive texts. That is an area that we think can improve as far as the overall networks are and that is where the UK is slightly behind some of the areas that we are seeing in other parts of the world. We are seeing a really strong ramp up in that reliability though and that is where we think the gain is going to be won between the operators. Speed is important but it is only important if you can get on and win the game of reliability, only then will certain operators have an edge over the competition.
Well it was announced that 4G was going to be rolled out amidst huge media hype, do you think it has lived up to expectations so far?
I think it does where it is, the caution is as I said the important point is that you have got to know that the media hype and the messages make it seem as though 4G is everywhere and it is not. It is not ubiquitous yet but everyday it is increasing with every operator EE have that edge because they have been out there for a year, O2 and Vodafone are making massive efforts to move quite rapidly and are moving very fast from our perspective and Three is right round the corner with a very solid underlying network which has been quite reliable and fast without 4G and we think they have a great foundation to build on to be able to roll out further from 4G.
The full report can be downloaded from the Rootmetrics website here: http://rtmet.com/1gLTSGv