[2 August 2008] Sam Crawford's Samknows website has been an oracle of reliable information about the UK's broadband scene since the early days of trigger campaigns and nascent, patchy services. Today the site contains a new addition to the plethora of broadband facts - the first results of his Performance Monitoring report.

During June and July this year the feedback from some 223 devices installed in people's homes was being collated, giving what must be the most detailed report into the performance of the UK's main ISP's ever published. Anyone feeling disappointed by the promises of "up to" broadband packages will understand that network performance is measured by more than just headline speed. The Samknows study has gone to great pains to identify and collect just about every metric which matters - the result is a fascinating insight into what happens under the hood of a broadband network.

The results are detailed and worth the effort to read - and in some cases contradict the received wisdom on many broadband forums. While more work is being done to allow firm conclusions to be drawn, the landscape is a lot clearer.

Sam Crawford should be congratulated!!

In the excellent 45-page report you can find an explanation of each metric in plain English, why its important, and how the various operators faired.

For example:

• Virgin's ADSL services suffered higher than average latency, especially during the evening, while their cable offering preformed well in this regard. Overall, Zen won the latency and packet loss tests.
• Measuring the ability to carry a quality voice call is more complex, and the tests reflect that. Here it is the turn of Virgin's cable platform to under perform, while their ADSL packages are in amongst the pack of providers. Atop the tables are Be and Zen - again.
• What about peer to peer?  All the operators performed pretty much equally with two notable exceptions - BT and PlusNet. Their traffic shaping and management policies appear to be limiting non-web traffic impacting P2P traffic like the BBC's iplayer. If you want to do more than just web surfing, then Zen performed the best.
• And the speed tests - more intelligent that many and carefully reported to ensure a fair and level playing field. The results show that on average you might expect to receive around 75% of the bandwidth you thought you were paying for. All of Virgin's products did poorly in these tests, largely due to the variations by time of day - not great in the evenings, in case you needed telling -  and while they also offer the highest quoted speeds (20 Mbps) you are likely to experience a smaller slice of their promises - only just living up to their bold advertising claims and perhaps a caveat for the DOCSIS 3.0 50Mbps services being planned. In the "meeting the promises" test, BT tops the list typically delivering 86% of the bandwidth they quote.
• Interestingly all providers performed much better when you tried to upload data, typically delivering over 80% of the promised bandwidth - but since the promises are not very generous it is perhaps unsurprising.