The term mission-critical is bandied about with wild abandon these days. Just about any application is "mission critical" and demands the "highest levels of availability." Yet, few applications really qualify for the label.
That cannot be said, however, about the Philadelphia Stock Exchange.
"When the market took a hit about a month ago, we processed 1.4 billion quotes in a day," says William H. Morgan, executive vice president and CIO of the Philadelphia Stock Exchange. "At its peak that meant 185,000 transactions per second sustained for four minutes."
To cope with such a heavy traffic volume and remain online without downtime, the Exchange uses V Series ftServers from Stratus Technology. These servers provide some of the highest levels of availability in the Intel Xeon server class.
"We are achieving better than 99.999 percent uptime availability using Stratus servers," says Morgan.
Founded in 1790, the Philadelphia Stock Exchange has a diversified product offering. It trades more than 7,000 stocks, 2,327 equity options, 16 sectors of index options and currency options, and futures.
The Exchange has been a Stratus customer for about 18 years. Until a few years ago, it used PA-RISC systems. The stock exchange has used several generations of these products up to the most recent Continuum line that ran the VOS operating system.
"Back then, it was a different world," says Morgan. "The big trend in securities is the shift to electronic trading."
Prior to 2004, the organization had three Stratus RISC-based servers. To cope with the realities of the digital world, the organization has had to expand its infrastructure. This includes Sun Fire servers running the Solaris operating system (mainly for interfacing with customer sites), as well as 22 Stratus ft Servers. It has also maintained one older Stratus Continuum box.
"Ft" stands for fault tolerant. The machine contains two complete servers that run in lockstep so that the OS and application see only one server, with both doing exactly the same thing at the same time. In the event of an error or malfunction, it continues to run unaffected by downtime or data loss.
"The ftServers support our main trading businesses and run a series of applications that we developed in house," says Morgan.
The Stratus ftServer V Series platform provided an easy way for the Philadelphia Stock Exchange to transition from Continuum to Intel-based servers.
"The options industry was in the process of a major shift to electronic trading, and we had to redesign both our business model and the technology supporting it in order to effectively compete in this new environment," says Morgan. "We relied on Stratus and its new V Series line to supply us with a system architecture that offered stability through its fault-tolerant design, speed and capacity, as well as flexibility to support our ongoing development efforts to add new applications in the future."
His company was among the first to use the V Series line, serving initially as a beta test site. He reports a smooth migration to the x86 boxes.
"We like mature products, and prefer others to kick the tires first before we adopt technology," says Morgan. "But in this case, we were one of the first as we had to move from a floor-based model to electronic trading with significantly more transactions per second."
The stock exchange has deployed mainly ftServer V 450 models. These systems form the backbone of the data center. It also has five ftServer 5700 models running the Red Hat Enterprise Linux 4 operating system for its front-end applications. The ftServer 5700 is a 2-socket, dual-core Xeon 2-way rack server designed for high transaction throughput and multi-tasking applications. It is a 2.8 GHz chip with 2MB L2 cache per core.
No Downtime Tolerated
As mentioned earlier, the Exchange can now cope with extended traffic bursts of close to 200,000 transactions a second. But Morgan doesn't expect to be able to rest on his laurels for long.
"Those transaction numbers are expected to increase as the options business is changing from nickel or dime increments to penny increments quoted," says Morgan. "When that takes place, our volume will surge again."
That's why he insists on paying extra for real fault-tolerant machines rather than attempting to load up with cheap white boxes. His customers demand no delays and that can only be achieved using high-availability servers.
"When customers notice a slowdown or problem with our electronic trading system, they route all their orders to another exchange, and we end up losing a lot of business," says Morgan. "Also, there is a federal rule that if you can't disseminate trades for 15 minutes, you have to shut down. So we have no room at all for downtime."
Although Morgan reports that Stratus V Series ftServers have done extremely well, he doesn't rely on only them to safeguard his valuable systems. The stock exchange has dual power grids, an interruptible power supply (UPS) and a diesel generator.
Part of the reason for these added layers of protection is an incident that happened in 1994. Philadelphia experienced severe ice storms that brought the power grid down for about an hour. This was before the stock exchange had deployed UPS or its current diesel generator.
"Our Stratus servers had battery-based backup built into them that was supposed to provide 20 minutes of backup power," says Morgan. "Yet those batteries carried us through the whole hour and we didn't lose any data."
The Philadelphia Stock Exchange plans to increase its processing power three-fold in the very near future. Its philosophy is to grow vertically via stronger CPU boards rather than horizontally by adding more machines. That is a better approach, says, Morgan, as you don't have to spread your applications too thin.
At the end of April, he says, Stratus is coming out with its next-generation server, and he expects these V Series machines to be four to five times faster.
"Stratus really gets what mission-critical is all about and is an exceptional company to work with," says Morgan.