Greg Young [MVP]

Sponsors

The Lounge

Syndication

Recent Posts

Tags

View more

News

Archives

Advertisement

Greg Young [MVP] » TCP: Buffer Management
Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
TCP: Buffer Management

So a long time ago I wrote some posts on buffer management in TCP servers (it might be worth going back and reading them as they explain why buffer management is important. There have been a few comments lately asking for more complete examples. Funny enough that's just what I have been working on lately. So here is the first of a set of code drops of some code that will be open sourced (consider it MIT/MSPL now) in its entirety (a nice little framework for writing scalable servers, TCP transports, etc). The project doesn't have an official name yet and is being run on our local svn so I will just upload a zip file for now.

 

You can download the source here: http://codebetter.com/files/folders/codebetter_downloads/entry181822.aspx

 

A quick run down of what is there.

BufferManager.cs - The main Buffer Manager class

BufferPool.cs - A class that abstracts a set of buffers to allow common operations

BufferPoolStream.cs - An adapter to the stream interface for a BufferPool

 

There are associated tests for these classes <> 75.

 

I know this seems like a lackluster post, but the code is worth going through. And if anyone has a good name for this library, let me know.

Posted 08-12-2008 2:14 AM by Greg

[Advertisement]

Comments

Oran wrote re: TCP: Buffer Management
on 08-12-2008 3:06 AM
The zip file appears to be corrupted or incomplete. How does this compare to the BufferManager in WCF? http://kennyw.com/indigo/51
Greg wrote re: TCP: Buffer Management
on 08-12-2008 3:14 AM

Weird I just downloaded it and it worked fine ... I will try uploading it again in the morning.

It is the same concept (full explanation as to why is in the post from a long time ago) ... The difference being it works with ArraySegment<byte> out of a big buffer .. and can be used with much smaller messages as well ... Also the BufferPool/BufferPoolStream are built on top of it to make things a bit easier to deal with.

Oran wrote re: TCP: Buffer Management
on 08-12-2008 4:09 AM
Hmmm, it worked fine when I downloaded the zip with Firefox, apparently IE is messing it up. Thanks for the explanation. When you say it "can be used with much smaller messages as well", do you mean "without unnecessary heap fragmentation due to pinning?" or is there more to it?
Greg wrote re: TCP: Buffer Management
on 08-12-2008 9:52 AM

Oran I should actually reflect their code before making that statement to insure my beliefs of it are correct.

karl wrote re: TCP: Buffer Management
on 08-13-2008 9:05 AM

There's a known bug for downloads through IE on codebetter. should get someone to look into it. I've always just hosted files in an alternative location

Joe R. wrote re: TCP: Buffer Management
on 08-13-2008 10:53 PM
Have you tested connections which involve real latency? I briefly looked through the code and it looks like the default buffer segment size is 1k. It's possible to mess up the thoughput on a TCP connection to a Microsoft client by buffering to programmer-friendly sizes. MS clients normally ack every-other MSS (not every other packet) so if you run out of buffer space and your bytes-in-flight don't match up with the MSSx2 you can introduce throughput problems. (Particularly if your outstanding buffer is less than the advertized TCP window size). You might take a performance hit using strange buffer sizes, but artificially slowing your throughput (in some cases pathalogically) will use up even more memory; I'm very interested in seeing if you avoided this problem. It's easy to miss, it usually only shows up if the conection has some latency involved. I'll try to test it out for myself, but see if you notice the the telltale sign of one smaller packet (on the wire) every six or seven packets in a large message.
Greg wrote re: TCP: Buffer Management
on 08-13-2008 11:14 PM

Joe see comment (in code) .... all my stuff allows you to give it a buffer manager so you can determine your own sizes ... I just kinda put that there as some sort of sane default for the uninitiated

Also my stuff doesn't send 1 buffer at a time it uses the BeginSend(ArraySegment<byte>) overload with N buffers which gets turned into the WSA overload with N buffers internally ... 1k is a good default so people can easily control their buffer sizes (i.e. 1k->20k etc)

Joe R. wrote re: TCP: Buffer Management
on 08-14-2008 8:24 AM
I noticed that you can change the size; I am interested in finding out if the default size causes the problems I am thinking of. I've seen it a lot from wire traces, usually from apps that wrote complex buffer management schemes. Like I said I'll test it out, it's likely that there is no problem. If there is a problem, tweaking the defaults or setting up some usage guidelines might improve throughput. It's a win-win.
Ray Akkanson wrote re: TCP: Buffer Management
on 08-14-2008 9:56 AM

The zip file appears to be corrupted or incomplete.

Ray Akkanson

Greg wrote re: TCP: Buffer Management
on 08-14-2008 3:38 PM

Ray it will be in public svn shortly, not sure why some have troubles downloading the zip.

Shaun wrote re: TCP: Buffer Management
on 08-19-2008 10:05 AM
If you pass in false to _AllowedToCreateMemory but pass an _InitialSegments > 0, you will get a UnableToCreateMemoryException. The test coverage doesn't seem to cover passing false to _AllowedToCreateMemory
Greg wrote re: TCP: Buffer Management
on 08-19-2008 1:02 PM

Sean thanks, good catch there was a test for the negative situation but not the positive. Here is the test for the positive situation and the associated code change.

       [Test]

       public void should_return_a_valid_buffer_when_create_memory_is_false_and_memory_is_available()

       {

           //BUG 1273: Buffermanager was not allowing construction when not allowed to create memory

           BufferManager manager = new BufferManager(1, 1000, 1, false);

           ArraySegment<byte> buffer = manager.CheckOut();

           Assert.AreEqual(1000, buffer.Count);            

       }

in BufferManager::ctor

           m_AllowedToCreateMemory = true;

           for (int i = 0; i < _InitialSegments; i++)

           {

               CreateNewSegment();

           }

           m_AllowedToCreateMemory = _AllowedToCreateMemory;

gOODiDEA wrote Interesting Finds: 2008.08.13~2008.08.21
on 08-21-2008 7:07 AM

.NETTCP:BufferManagementASP.NetLoadTestingandOptimizationToolkit-Soyouwanttobeahe...

gOODiDEA.NET wrote Interesting Finds: 2008.08.13~2008.08.21
on 08-21-2008 7:08 AM

.NET TCP: Buffer Management ASP.Net Load Testing and Optimization Toolkit - So you want to be a hero

Buffer Management for Networked .NET Applications « Roman’s Blog wrote Buffer Management for Networked .NET Applications &laquo; Roman&#8217;s Blog
on 11-11-2008 5:55 PM

Pingback from  Buffer Management for Networked .NET Applications &laquo; Roman&#8217;s Blog

Bijan wrote re: TCP: Buffer Management
on 12-06-2008 2:00 PM

Greg,

You may want to consider using ReaderWriterGate Lock (msdn.microsoft.com/.../cc163532.aspx) from

Jeffrey Richter’s Power Threading Library (wintellect.com/PowerThreading.aspx)

instead of lock inside CheckIn...

Regards,

Bijan

Greg wrote re: TCP: Buffer Management
on 12-19-2008 12:26 AM

Bijan yes that is a possibility

generally however I would prefer to use a lock free stack :)

Cheers,

Greg

Obin wrote re: TCP: Buffer Management
on 02-23-2009 7:44 PM

Zip is still corrupt.

Any alternative location?

gaksd wrote re: TCP: Buffer Management
on 03-14-2009 2:15 PM

ReaderWriterGate is useless, it has an extremelly hot path. The entire PowerCollection is just an experiment, I'd avoid it at all costs.

Tried downloading the zip and RAR can handle it but contents are not readable.

Any fix, just RAR it :)

John Nagle wrote re: TCP: Buffer Management
on 05-18-2009 7:17 AM

Hi Greg...it seems that the download link is broken...it takes you to a download page, but when you click "download", it just takes you to a page that says "Object moved to here".  The link to "here" is a link to that same page.  :-)

I'm new to async sockets in C#, and have found numerous small examples, including the samples on MSDN, that don't properly demonstrate using BeginSend within the callback to finish sending data.  So naturally I was very happy to find your blog.  :-)

-John

Add a Comment

(required)  
(optional)
(required)  
Remember Me?