This experience led to an idea -- could we utilize electronic test equipment, of the type used by product engineers, to run large-scale benchmark tests in an isolated, repeatable environment? And would the results of such tests be indicative of what we would see in the real world?
Well, I finally did the first series of tests along these lines, and the results can be seen in the Farpoint Group Technical Note available here. We tested Aruba Networks Inc. (Nasdaq: ARUN) vs Meru Networks Inc. in a number of dimensions, and the real-world results showed an excellent correlation to those obtained from the test equipment, in this case a VeriWave Inc. WaveTest 90. While this first study is hardly definitive, I’m encouraged enough to dub the concept of testing in isolation virtual benchmarking. And I think enterprises will eventually use this technique for head-to-head evaluations in place of real-world testing. Sure, the equipment involved needs to be useable by mere mortals, and the RF channel modeling needs to be more robust, especially as MIMO and .11n take center stage. But the potential savings in time and hard currency is more than inviting.
And, in case you’re wondering, Aruba kicked butt in the tests. Those guys continue to amaze me.
— Craig Mathias is Principal Analyst at the Farpoint Group , an advisory firm specializing in wireless communications and mobile computing. Special to Unstrung