Shocking Results in LR's SSL VPN Test

Light Reading has completed its biggest ever technology test, with eight (8!) SSL-based VPN gateways put through their paces. And the findings will be like gold dust to any network manager with even a passing interest in VPN technology (see SSL VPNs: Access Anywhere, Anytime ).

Our testing partners, Spirent Communications and Network Test, got their hands on the leading products from the following vendors:

• Array Networks Inc.
• Aventail Corp.
• NetScaler Inc.
• NetScreen Technologies Inc. (Nasdaq: NSCN)
• Nortel Networks Corp. (NYSE/Toronto: NT)
• PortWise AB
• Symantec Corp. (Nasdaq: SYMC)
• Whale Communications Ltd.

The notable absentee from that list is Cisco Systems Inc. (Nasdaq: CSCO), which only entered the market as the tests were nearly complete (see Cisco Takes a Stab at SSL).

The outcome? A mixed bag. Each of the vendors' gateways has a core strength, but there is no clear winner in terms of overall functionality, and there is still plenty of work to be done to improve each of the products.

Certain products excelled in individual tests, and the report offers a crucial breakdown of which products came out on top in each area, along with an extensive analysis of the results.

Aventail, for example, offers the broadest range of configuration options, while NetScreen's user interface was commended for its clarity. NetScaler set the pace in scaleability, but Nortel performed exceptionally well in the tests of server-based systems.

The gateways were subjected to a range of performance and scaleability tests to determine which products performed best in terms of: session rate; concurrent user capacity; Microsoft Outlook Web Access (OWA) handling (with and without denial-of-service attacks); forwarding rate; and security.

The overall findings of the test are:

Secure Sockets Layer VPNs are generally scaleable, handling from hundreds up to tens of thousands of concurrent users per gateway, with NetScaler’s NS 9500, which supports up to 58,000 users at one time, top of the pile.

While all the tested systems support OWA, there’s lots of variation in how well they do so, with some response times slowing to minutes when there are a lot of concurrent users.

The best performers were those with hardware-based SSL acceleration, with some pushing data at an impressive 500 Mbit/s, while those performing encryption/decryption tasks in software ran as slowly as 1 Mbit/s.

The products tested were generally secure, with only a few wrinkles, mainly involving weak default configurations, uncovered.

The results, according to David Newman, president of Network Test, provide vital guidance to users because of the unique methodology deployed. "We hope our tests help enterprise network architects make informed decisions in terms of features, performance, and security,” says Newman. He says the test attracted a lot of participants because "we created a new methodology, one that includes what is arguably the killer app for remote access: OWA, the Webified version of Microsoft Outlook."

And it's a methodology that has legs. "This is a milestone test in that it establishes an industry standard methodology for testing SSL VPNs,” says Mark Fishburn, VP of technical strategy at Spirent Communications, who describes the test as "the most comprehensive SSL VPN test ever performed in this competitive market segment."

And why are these products important? Because SSL VPNs, which enable access from almost any Web browser, look to offer a very neat complement to IPSec VPNs, as they are very well suited for remote access and extranet applications, which have been sore spots for IPSec.

— Ray Le Maistre, International Editor, Boardwatch