by Don Gardenhire, HomeGrid Forum
All test plans are not created equal, so forget the hype about throughput, and make sure your testing takes real homes and real environments into consideration
I drafted this as an email to a service provider in response to recent publicity regarding the spate of testing the Homeplug Alliance is having done with their own version of a test plan that I find too limited to reflect the real world.
I believe this tells the real story and I have taken the liberty of quoting from two separate articles, the first from “The Online Reporter” that touts HPAV2 as being faster (this article published an update the following week), the second from SmallNetBuilder, that provides a more complete view of more difficult circuits that will be found in many homes.
Dear service provider,
If you don’t want truck rolls, you need to understand The Online Reporter’s recent articles (especially issue 936). It gives a glowing review of HPAV2, having done extensive testing. On the face of it there is no contest, but it is only by reading between the lines that you will see what it really proves and that, in fact, it is G.hn that will ultimately save you millions in truck rolls and easily deliver video where needed.
There will be a lot of publicity in the next weeks generated from a test plan published on the HomePlug Alliance web site which encourages press to execute and report on it. The test plan requests a number of preparatory steps, including:
• Disconnecting the WAN
• ONLY testing one link at a time
• NOT using UDP
• NEVER considering error rate
• NOT using a surge protector
• NOT allowing noise generators like phone chargers
• DO use open source free traffic generators
• ONLY consider bandwidth – not QoS or Multi-Node/Multi-traffic
In other words, don’t use it like a consumer would!
“The Online Reporter” faithfully duplicated the test plan from the HP Alliance and executed a performance test as suggested by the test plan to get the results published in the article. This review shows HPAV2 beats G.hn in simple throughput in their outlets; but there are two more important hidden gems that need to be highlighted to understand why G.hn will actually save service providers millions and why I’ve said from the start that throughput is actually the least important metric in power line testing.
First is this quiet little paragraph at the back of the article. I knew immediately upon reading it that G.hn was the one that connected, but it was left out of the conclusions. I did contact the writer to be sure, and “YES!” it was G.hn that connected before the lines were “conditioned”(this was also shown in the following update). Here’s the paragraph in question:
“When we began testing we found that results were very, very poor, especially the master bedroom where one technology could only attain speeds of 20 or so Mbps and the other could not even connect. A typical consumer might not have taken the next step.”
The Online Reporter is right; a consumer would not have taken the next step. He would have used the G.hn and returned the HPAV2 to the service provider. Keep in mind that this G.hn connection rate would have delivered any video stream available today without issue and avoided a truck roll.
The second item I noticed was the size of the home. The longest distance from any adapter (tested one link at a time) was 28 steps. Tiny house to be sure. The point a service provider must consider is that – although the G.hn device may not have had the high throughput after a technician was hired, remediated the home wiring, and was paid – a normal consumer would have used it gladly while returning the HPAV2 product. Seriously, would a service provider roll a truck for every house to remove disturbers or change circuit breakers?
The SmallNetBuilder article, published on July 1st 2015, is in my view a more technical comparison of power line that considers conditions not allowed for in the test plan performed by The Online Reporter. It also proves what I’ve been saying is the case from the start:
“When G.hn and HomePlug AV2 adapters collide, both networks will suffer throughput loss, but HomePlug AV2 will suffer more than G.hn.”
Even more truck rolls for HPAV2 in the MDU market! AND this is what happens for G.hn in the tough loops! Again quoting SmallNetBuilder:
“This is actually the best throughput we’ve ever measured in Location E for any powerline adapter tested!”
The weeks ahead will be interesting – we are about to publish our own guidelines for how to test and achieve meaningful results (see our blog on July 13th). We are happy to challenge anyone to run our rigorous tests because we know the quality of the end result!