Where & Why were the tests conducted?
On Sunday 01 August 2010, myself & a good friend set out around several different locations, mainly around Hamilton, Cambridge & in between, undertaking 3G Data Speed Tests on both Vodafone NZ & Telecom’s XT Mobile Networks. My friend Chris, owns both of these devices & I must say, it was quite enjoyable playing with these devices over the course of a few hours.
The tests were conducted to provide interested people an indication of 3G Data Speeds at the various locations, where the tests were completed, on the Vodafone 3G & Telecom XT Networks & specifically the Download Speed was being measured here, due to the disparity with the hardware limitations. These tests are a “snap shot” only & Speeds will vary depending on the time of day, number of users & backhaul to each Cell Site, for both Networks. Whilst I appreciate that downloading a 20MB file provides more accurate data throughput information for HSDPA Networks, this was not possible to do on the day of the testing.
The Telecom XT Network is relatively new (just over 1 year 2 months old, at the time of writing this article), compared to Vodafone’s 3G Network (which has been active since 2005) & upgraded with Speed enhancements over time. As a result, it is understood that Vodafone have a greater number of 3G customers, than what Telecom’s XT Network currently has, therefore – due to the greater load on the Vodafone Network, Telecom’s XT Network does have an advantage when it comes to 3G Data Speed Testing, at this time.
What frequencies does Vodafone NZ & Telecom XT Network’s support?
For those who don’t know, Telecom’s XT & Vodafone NZ’s Mobile Networks operate on different frequencies. In a nut shell, the below gives an overview of this:
Vodafone NZ Network
- Nationwide 900MHz GSM 2G Network with infill 1800MHz GSM to provide extra capacity to major cities
- Metropolitan WCDMA 2100MHz 3G Network covering all major cities and some towns with WCDMA 900MHz for Rural coverage
Telecom XT Network
- Nationwide WCDMA 850MHz 3G Network with infill WCDMA 2100MHz network to provide extra capacity in major cities. Telecom NZ does not have a 2G Network.
The tools of the trade….
Apple iPad
We used the Apple iPad 3G 64GB, using Firmware Version 3.2 to do our testing on the Telecom XT 3G Network & the Apple iPhone 4 using Firmware Version 4.0.1 to do our testing on Vodafone’s 3G Network.
It should also be noted, that there are some significant differences with each device. The iPad supports Data Speeds up to 7.2Mbit HSDPA download speed, but only a MAX of up to 384KB/s upload speed, utilising WCDMA 850, 1900 & 2100Mhz 3G frequencies & GSM/EDGE 850, 900, 1800, 1900 MHz for 2G Network support. The iPad does not currently support HSUPA for faster upload speeds, nor does it support WCDMA 900Mhz , which is what Vodafone NZ utilise in Rural areas, as previously explained.
The lack of WCDMA 900Mhz support for the iPad I found particularly interesting, as both the iPhone 4 & iPad were released at a very similar time, not to mention no HSUPA support either for the iPad. Both devices are sold at a premium price in New Zealand.
Apple iPhone 4
The iPhone 4 supports Data Speeds up to 7.2Mbit HSDPA download & also up to 5.8Mbit HSUPA upload speeds. The iPhone 4 has a Quadband Chip operating on WCDMA 850, 900, 1900 & 2100Mhz 3G frequencies. iPhone 4 also supports GSM/EDGE 850, 900, 1800, 1900 MHz for 2G Network support. 900Mhz WCDMA support is great news for users of the Vodafone 3G Network. The previous iPhone 3G & 3Gs did not support 900Mhz WCDMA, meaning you would drop down to the slower 2G GPRS Data Speeds, outside of the main cities/towns – which is roughly equivalent to dial-up speeds.
Now users will get 3G coverage wherever Vodafone have coverage, with the iPhone 4 & both Telecom XT & Vodafone NZ advertise 97% coverage of where New Zealander’s live, work & play, this is not based on land coverage of course.
How were the tests conducted?
The 3G Data Speed Tests were all conducted in exactly the same location for each device & the best of three tests were recorded for each location. A grand total of 15 Speed Tests were conducted & due to the weather (light rain) all tests were conducted inside the car, to protect these very expensive devices. The tests were conducted between 1300 – 1700 (1:00pm-5:00pm) on Sunday afternoon, the 01 August 2010.
Network Latency – “Ping” times
I have found through previous Speed Testing that I have conducted, that the Speed Test App does not report accurate ping times & these should be taken with a large grain of salt, in my honest opinion. In the past I have used Command Prompt on a Windows PC, which has shown latency of around 80-130ms on average on the Vodafone 3G Network. I have not personally tested latency the Telecom XT Network, using this method.
Ok, now that I’ve got all the technical information out of the way, let’s get into the good stuff, that you’ve been waiting to see – the Speed Tests Results!
Location, location, location……
Eagle Way – (Across from The Base Shopping Centre) Hamilton
Vodafone NZ
Telecom XT
Rototuna – Hamilton
Vodafone NZ
Telecom XT
Five Cross Roads – Hamilton
Vodafone NZ 
Telecom XT 
Hillcrest – The Warehouse Shopping Centre – Hamilton
Vodafone NZ 
Telecom XT 
Waikato University – Hamilton
Vodafone NZ 
Telecom XT 
Silverdale – Hamilton
Vodafone NZ 
Telecom XT 
Matangi – Rural Test
Vodafone NZ 
Telecom XT 
Central Cambridge – Victoria Square
Vodafone NZ 
Telecom XT 
Leamington Domain – Cambridge
Vodafone NZ 
Telecom XT 
Cambridge KFC
Vodafone NZ 
Telecom XT 
Speed Test - Undertaken @ 100km/hr in a car between Tamahere & the Hamilton City Boundary
Vodafone NZ 
Telecom XT 
Hillcrest New World – Hamilton
Vodafone NZ 
Telecom XT 
Hamilton CBD
Vodafone NZ 
Telecom XT 
Frankton Railway Station – Hamilton
Vodafone NZ 
Telecom XT 
Saint Andrews – Hamilton
Vodafone NZ 
Telecom XT 
Conclusion
Telecom’s XT Network was found to be faster for 10 out of the total 15 Speed Tests that were conducted on Sunday 01 August 2010. Some results, were twice as fast or more than the Vodafone NZ Network, at the time of testing.
Interesting though however, Speed Testing in the smaller townships like Cambridge & Matangi – Vodafone NZ did have faster 3G Data throughput with 3 of the 4 tests in these specific areas. With Matangi in particular, Vodafone NZ was twice as fast as Telecom XT, however this was certainly not the norm. Vodafone NZ was also faster at Eagle Way – across from The Base Shopping Centre & finally in Silverdale – rounding off the total of 5 areas that Vodafone were faster, after that, it all went down hill for Vodafone.
During the testing – I made some interesting observations with the iPhone 4, compared to my Nokia 6121c, also on the Vodafone Network. For instance, arriving into Cambridge, my Nokia picked up full 3G signal, 1 minute 30 seconds before the iPhone 4 hit full 3G signal (both phones were held in the same position & my hand was not touching the grip-of-death area on the iPhone 4). Furthermore, when attempted it was “easy” to replicate the “grip-of-death” (signal bars dropped from 5-2 within ~20seconds) with the iPhone 4 & with direct comparisons with my Nokia 6121c throughout the day, the iPhone 4′s signal strength would definitely fluctuate quite a bit, whereas the Nokia would remain very stable in comparison.
Apart from Matangi, where both Vodafone NZ & Telecom XT had patchy coverage (whilst travelling to the testing area), the iPad had rock steady signal strength throughput the various locations where testing was undertaken.
So, that brings this round of 3G Network Speed Testing to an end, I hope you enjoyed the read & feel free to leave a comment or go out & do some 3G Speed Testing in your own area! 
Pingback: Tweets that mention 3G Data Speed Tests – Telecom XT on iPad vs Vodafone NZ on iPhone 4 | Grantisnz's Blog -- Topsy.com
Pingback: Twitted by Aaroona
Hi Grant,
First off, good post mate.
It’s interesting to see where you got speed and where it was a wee bit slower.
I am quite surprised to see such speeds on Vodafone since where I am, its a regular running joke within the group of people I come into contact with, that Vodafone is slower than dial-up.
Take today for example – We weren’t able to achieve anything over 50KB/s (.56mbps).
TelecomXT are the only ones that are able to provide consistent speed AND coverage in my area over 3G. It’s not uncommon to reach around 2.5mbps here.
It will be interesting to see if you go back and do the same tests, if you use similar hardware for both providers, whether the upload speeds with be similar (since the XT ones were restricted in this test, sadly).
But at this point in time, it seems that my results have had a similar output to yours. It just so happens that currently, in my area, XT provide the fastest 3G in my situation. So the decision is quite clear now.
Cheers.
-Aaron
Did you do multiple tests in each location? It appears you used one device per carrier, are you testing performance of the speedtest application on the device or the network? Do you know how much capacity and headroom Enternet Online provides to the speedtest instance or has in their connections towards Telecom and Vodaphone?
You were correct, downloading a file from a KNOWN location might have actually been useful. You have wasted your time and petrol.
Hi Anonerd,
Thanks for the reply.
Now to answer your questions…
As stated in my blog - “The 3G Data Speed Tests were all conducted in exactly the same location for each device & the best of three tests were recorded for each location”“.
I thought it was quite obvious that the devices were being tested on each network, plus the title of the blog kind of gives that away as well “3G Data Speed Tests – Telecom XT on iPad vs Vodafone NZ on iPhone 4″. Clearly the devices were not being used to “test the speedtest application” the number of tests conducted, was intended to provide a comprehensive overview of the network performance for each device.
I have no idea about how much capacity and headroom Enternet Online provides to the Speed Test & in any case – that falls outside the scope of my testing.
I completely disagree with your comment about wasting my time & petrol. As stated it was not possible to download ~20MB files, at the time of my testing, perhaps in the future it will be. The tests a “snap shot only” & for all intensive purposes, the testing met the objective.
My point is that you are reporting on network performance using rather flawed methodology and drawing a conclusion about which network is faster.
By limiting your test of each network to one device you don’t know if you are testing the network itself, the performance of the devices network interface or how well the speedtest app runs on the device. Memory and processing capabilities can have a marked impact on speedtest results. Additionally speedtest.net allows anyone to run a test server with no requirements on connectivity. EOL are a very small provider who probably don’t have large or diverse connections. Your tests will likely be affected by EOL’s capacity, who they buy from and their upstream(s) connectivity towards vodafone and telecom.
In short, the moment you decided to use speedtest.net any useful conclusions went out the window and by limiting each device to one network you cant possibly claim to have identified which network performs better.
Ok Anonerd,
I don’t see my methodology as being flawed. I clearly stated the purposes of my testing & mentioned that the testing was a “snap shot” – thereby giving an indication of the network speeds, for the devices that were used, at the given time the tests were undertaken.
I did not have *ANY* other devices to test with – my Nokia 6121c needs to be tethered to a laptop (for Speed Testing), but we didn’t have a laptop to do this with, at the time of testing.
Both the iPad & the iPhone 4 use the Apple A4 1Ghz CPU chip. The iPad is believed to have 256MB RAM & the iPhone 4 512MB RAM, again these specs are very similar & as already mentioned, both devices support up to 7.2Mbit download speeds. So I can’t see any limiting factors here.
Now – I would like to correct you on some comments about EOL & also your remark “Additionally speedtest.net allows anyone to run a test server with no requirements on connectivity“.
Can I ask where you got this information from?? Because I can tell you, this could not be further from the truth. A quick & easy look on the SpeedTest.Net website states:
“Setup is easy and the requirements are a web server and an Internet backbone connection with at least 100 Mb/s available“
Link for above quote: http://www.speedtest.net/host.php
Additionally, I have spoken to an EOL employee this afternoon & I expect some formal reply from them, to back up the legitimacy of using SpeedTest.Net. Also, as recommended when doing Speed Tests – it is advised to use the closest server & the Tauranga server was the closest server for all tests.
Finally – many people use Speedtest.Net to gauge network speed for both landline / wireless & mobile networks, including but not limited to:
Chris Keall – well respected Technology journalist for the NBR magazine, here is an example of his work: http://www.nbr.co.nz/opinion/chris-keall/guess-what-xt-s-upgrade-rocks
Um, you really need to run the tests on the same device.
iPad does not support HSUPA which explains the apparent slower upload speeds on XT. You need to do those tests again with iPhone 4.
Additionally iPhone 4 has a completely different cellular chipset (Skyworks) to iPad (Infineon) so the performance is likely to differ.
Hi Jimbo,
Thanks for the reply.
I clearly acknowledged the device limitations & differences & I stressed that the tests were done for measuring Download Speed. As mentioned, both devices support up to 7.2Mbit download.
The upload Speed advantage did not help the iPhone 4 on the Vodafone Network IMHO, as XT still won 10 out of the 15 tests. It was not possible to switch microSIMS between devices for every test, due to time constraints & the hassle of doing so.
Regards,
Grant
“It was not possible to switch microSIMS between devices for every test, due to time constraints & the hassle of doing so.”
BEACHED AS BRO!
The testing took 4 hours as it was & we had other things to do – for all intensive purposes, it was not necessary to switch SIMS – because XT still would of spanked Vodafone anyway IMHO.
How about you do your own tests??
I did see the comment about download speed however the upload speed will affect the download speed. I never said Vodafone was short changed by your test, in fact I believe the XT tests would have been faster if you had done them on the iPhone 4 rather than the iPad.
My comment on the differing cellular chipsets still stands. The Infineon chipset in the iPad does not work as well at lower signal levels as the Skyworks one in the iPhone 4.
Things such as latency also come into play with short tests such as the speedtest.net ones you have done, and having the different chipsets are likely to introduce differences there as well.
I guess my gripe is that it isn’t much of a comparison when you are changing two variables. I would have been happier with a single network tested on both devices or differing networks on the same device. But differing networks on different devices is hardly a comparison.
@ Anonerd
I just got a reply from EOL:
Hi Grant,
Mark has asked me to respond to you with information about EOL’s network, and in particular speedtest.net and the comments on your blogg from Anonerd.
The only thing this person got right is yes we are a small company.
However, our bandwidth to the outside world until recently provided us with an excess of 100Mbps over and above our maximum use, this has now been increased to over 1Gbps. The reason we need such a high amount of spare bandwidth is because of our specialised customer base which is predominantly business, Government and Corporate to whom we have to allocate a guaranteed speed even though they never use it. Hence we have more spare bandwidth available than most other ISP’s. You can easily see this by running speedtests to all the NZ speedtest.net sites from a nice fast connection and you will see that EOL is usually the fastest.
In addition EOL has this bandwidth spread over more than one upstream circuit, and to more than one upstream provider. OOKLA the owners of speedtest.net chose EOL as their only NZ beta test site for 3 months before launching it because we where the fastest and most reliable of all their NZ mirrors.
Small does not always mean slow or inferior, so I would say that your 3G tests are accurate and we are grateful that you chose our server to perform them on.
Kind Regards,
Terry Coles
–
‘As Fast As You Want To Go’
EOL Wireless Broadband
http://www.eol.co.nz
Tel: +64 7 5770066
Fax: +64 7 9286063
I’ll concede that EOL are larger in a network sense than I had assumed but I very much doubt they provide even 100Mb of dedicated bandwidth to the test server, what would be interesting is if you could get them to give you real capacity and utilization data. Regardless, you will never convince anyone who knows about measuring networks (journalists generally do not fall into this category) that using speedtest.net is worthwhile.
Yes – I’ve noticed that you “assume” far to much…..
IT IS a requirement of speedtest.net to have 100Mbit available for companies who host speedtest.net servers – stop trying to distort the FACTS!
The tests ARE accurate (& EOL have backed me up on this) your just clutching at straws to try & undermine my testing.
I have no idea why you are trying to disprove the testing, because at the end of the day – the data is accurate.
Good testing. Back to back it sure looks like XT rocks.
Pingback: Speed tests show that XT still leaves Vodafone in the dust on the iPhone 4 despite 900Mhz capability | iPhonewzealand
Pingback: [S P E E D! Report] VodafoneNZ vs Telecom XT in Hamilton, New Zealand | Grantisnz's Blog