Thursday, March 19, 2009

Software is Low Latency's Weakest Link...Unless You're Running Red Hat Enterprise MRG

There's an article in today's Wall St and Technology, Software is Low Latency's Weakest Link.  The article states:

The weakest link in the low-latency value chain is older software or poorly written code, not market data feeds, lack of ultra-fast processors or older networks, according to experts at Wall Street & Technology's Accelerating Wall Street 2009 conference.
"Within the applications, we see the greatest opportunity to improve latency," said Rob Wallos, global head of market data architecture at Citi during the session The Complete Low Latency Value Chain. "Hardware can give you a generic 20 percent improvement in performance, but there is only so far you can go with hardware. "
This is certainly true in general.  We have found that even moving up to exotic hardware like infiniband only provides incremental gains in performance unless you optimize your application to take advantage of that hardware.  That's why we've been working with hardware manufacturers and optimizing Red Hat Enterprise MRG for the latest hardware as well as for Linux.

For example, MRG includes RDMA drivers to achieve extremely low latency on infiniband.  Compare the results of MRG Messaging running on infiniband with standard TCP versus with our RDMA drivers:

This chart illustrates latency using 1K messages at a sustained throughput of 50,000 messages/second using MRG Messaging.  This is for fully reliable application-level latency between three peers: a producer is sending a message through a broker to a client, and that message is acknowledged back.  So, there are four network hops in this case.  As you can see with standard TCP (the red line), the latency is nothing ground-breaking at around 1 millsecond, even though we're using infiniband.  In other words, moving the application to infiniband doesn't automatically improve latency significantly.

With our RDMA drivers , however, MRG can achieve latencies an order of magnitude better--about 70 microseconds on this same particular hardware.  And, remember, this is for 1K message sizes and 4 network hops!  So, yes, it is true that in general, low latency bottlenecks are in software.  However, we have been developing MRG to take full advantage of modern hardware and also Linux, so we've pushed the bottlenecks back to the hardware domain in many cases.

I should also note that our next release of MRG will include a cross-memory driver for co-locating our messaging broker with an application client.  This would remove one of the network links for latency and should allow us to halve our latency performance.


Unknown said...

InfiniBand is "exotic hardware?" Go tell Larry Ellison that his HP Oracle Database Machine and Exadata Storage server (all running InfiniBand) is exotic hardware...let me know what he says...

Rmush said...
This comment has been removed by the author.
aRman said...

could you please help me understand how are you counting the "4 hops" while measuring latency? are these just the application servers, or does the path between the application server and the network switch count as well?

Bryan Che said...

These are the 4 network hops:

producer --msg--> broker
producer <--ack-- broker

broker --msg--> consumer
broker <--ack-- consumer

Judy Misbin-May said...

I think that Network is your hot spot. Most top traders are within microseconds in the algo models - that leaves them to the one who gets there fastest. Check out to learn more about the importance of the network in the low latency race. See the White Paper

Anonymous said...

gucci replica handbags
men gucci shoes
Gucci men sneakers
Gucci men moccasins
gucci women sneakers
gucci women boots
Gucci men boots
Gucci shop
Gucci bags
Gucci shoes
wholesale gucci shoes
cheap Gucci handbags
Gucci ON sale
Gucci Belts
Gucci small accessories
Gucci hats & scarves
Gucci wallets
Gucci Handbags
Women Gucci shoes
Men Gucci shoes
discount gucci shoes
cheap Gucci shoes

Unknown said...

The Tax Return Crack-Up<3>
Granted, there are usuallyMicrosoft Office 2010write-ups when presidential contenders make their tax returns available, but the coverage falls far short of the Office 2010
full court press (pardon the pun) that the Clintons have received. What's Microsoft Office 2007different now?Office 2007One possibility is that most upper middle class Democrats, and therefore most Microsoft OfficeOffice 2007 keyeditors and reporters of our nation's big papers as well as Office 2007 downloadtelevision producers, are Obama supporters who think that Hillary should hurry up Office 2007 Professionaland drop out of the race already.Microsoft outlook
Microsoft outlook 2010Whom elite liberals are pulling for really does shape political coverage in ways

Unknown said...

So I would certainly take care of Supermarkets similar Diablo III itemsto Acme, in spite of WalMart's size. My check out is very that being employed by the government doesn't make a tangible tendency. It will be possible the very same would be genuine intended for some other buyers (elizabeth.h., condition govts). My partnerBuy Guild Wars 2 Gold and i researched an associated style in a very 3 years ago article from the Harvard Rules & Insurance plan Critique: