Companies that develop browsers have their own Web browser benchmarks touting how much better their score is than the competition. The question of bias is always brought up when a new benchmark is announced for the latest version of a browser.

Finding an independent browser benchmark is difficult, and the goal of RoboHornet was to get rid of any bias so that market driven developments could dictate the evolution of browsers. However, Google (News - Alert) is the company responsible for the creation of RoboHornet and any hopes of getting rid of any bias was quickly diminished. The validity of the benchmark was further tested when Microsoft (News - Alert) claimed the benchmark was not suited for real world performance, even though it ranked first by the new test with IE10.

RoboHornet has a committee of developers and industry representatives from key players including Facebook (News - Alert), Microsoft, Mozilla (News - Alert) and Google. By including members from different organizations, the expectation is change will be made after a consensus is reached that will benefit everyone.

Regarding the great score for IE 10, Microsoft’s director of Internet Explorer marketing Roger Capriotti wrote in his blog, “While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn’t all that representative of the performance users might encounter on real-world sites. Like all micro-benchmarks, RoboHornet is a lab test that only focuses on specific aspects of browser performance.”

Image courtesy of Shutterstock.

This is the problem RoboHornet was trying to eliminate because most of the benchmarks created by vendors will favor their own products. So even the test Microsoft conducted using its own RoboHornetPro resulted in a much better result compared to Chrome.

Hopefully the back and forth will eventually get settled. In the meantime, it is important to note the speeds they are measuring are differences in milliseconds. While this might be relevant for developers, the average user will not notice a 10 millisecond advantage over another browser.

Anyone can run the benchmark to see how a particular browser performs. It can be customized by checking and un-checking the option you would like to test. It uses a novel scoring system to show the overall score and based on the score the community proposes solutions. The score is based on how major browsers perform. It is compared to scores running with the same hardware and version of the benchmark and the overall score is called the RoboHornet Index. The index measures the result based on a comparison of stable browsers on average hardware. If the index is under 100 it is below average, 100 means it is in the middle and above 100 is better than average. RoboHornet is open source and it is currently in early alpha state.

Want to learn more about the latest in communications and technology? Then be sure to attend ITEXPO Austin 2012, taking place Oct. 2-5, in Austin, TX.  Stay in touch with everything happening at ITEXPO (News - Alert). Follow us on Twitter.