The study was performed in Berlin, measuring typical commuter trips (lasting between 30 and 40 minutes), and compared five different navigation systems:
- The “Bosch Navigation” App Ver.1.2.21, running on an Apple iPhone 4 – the software had access to real-time information provided by INRIX called “Inrix Traffic”.
- A Garmin nüLink! 2340 device as a genuine PND; this device had real-time traffic information provided by Navteq called “3D Traffic”.
- The “Google Navigation” App running on a HTC Desire device with Android 2.3.3; this device, too, had real-time traffic information, albeit it is not clear to what degree it is being used in the computation of the routes.
- TomTom HD 1000, a genuine PND with online access to TomTom’s HD traffic service.
- TomTom XL which is also a genuine PND, but without access to online traffic flow information.
“The results of this campaign show, that there is a roughly 12% difference in travel time between the best and the worst PND. This difference is not very large but statistically highly significant. The closer analysis in section 3 show, that the mechanism responsible for this difference is the fact that the TomTom devices sometimes found routes that were considerably better than what the other devices found. This seem to happen in about 25% of all the cases, while in the remaining 75% the difference in the travel times is fairly small. In addition, those different routes also display a shorter waiting time, again with the gain in waiting time being in the same order of magnitude as the overall gain.“
Routing Algorithms vs Traffic Information
The problem of this type of study is that it mixes routing algorithms with traffic information and the quality of the integration between the two components. Therefore INRIX and NAVTEQ are likely to answer to TomTom’s claim that this type of benchmark do not represent them fairly as it strongly depends on the implementation of their traffic feeds in third party devices and apps.
Comparing different traffic information feeds is indeed a complex task. It usually requires to use historical logs from traffic providers and compare them with the reality in the field via GPS logs recorded from real drivers.
The international Traveller Information Service Association (TISA) is currently working on a standardisation method, derived from QBench, a tool currently used by BMW.
The TISA “Traffic Flow QBench” Task Force, chaired by Carsten Lux (BMW) is aiming at “building a standard tool for independently assessing the quality and accuracy of “Traffic Flow Information” provided by different service providers.“
Once the standard is in place it will be easier to make assumption on the quality of each provider; until then we are likely to see and hear a lot of marketing based on “independent“ studies.
Download the full study below
DLR Study HD Traffic.pdf