TCP: Buffer Management

So a long time ago I wrote some posts on buffer management in TCP servers (it might be worth going back and reading them as they explain why buffer management is important. There have been a few comments lately asking for more complete examples. Funny enough that’s just what I have been working on lately. So here is the first of a set of code drops of some code that will be open sourced (consider it MIT/MSPL now) in its entirety (a nice little framework for writing scalable servers, TCP transports, etc). The project doesn’t have an official name yet and is being run on our local svn so I will just upload a zip file for now.

 

You can download the source here: http://codebetter.com/files/folders/codebetter_downloads/entry181822.aspx

 

A quick run down of what is there.

BufferManager.cs – The main Buffer Manager class

BufferPool.cs – A class that abstracts a set of buffers to allow common operations

BufferPoolStream.cs – An adapter to the stream interface for a BufferPool

 

There are associated tests for these classes <> 75.

 

I know this seems like a lackluster post, but the code is worth going through. And if anyone has a good name for this library, let me know.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.

19 Responses to TCP: Buffer Management

  1. tintin04 says:

    Download is broken, please reupload this useful code again.

    Thanks

  2. tintin04 says:

    we cant download buffer management from the link http://codebetter.com/files/folders/codebetter_downloads/entry181822.aspx

    Please reupload it again

    Thanks

  3. John Nagle says:

    Hi Greg…it seems that the download link is broken…it takes you to a download page, but when you click “download”, it just takes you to a page that says “Object moved to here”. The link to “here” is a link to that same page. :-)

    I’m new to async sockets in C#, and have found numerous small examples, including the samples on MSDN, that don’t properly demonstrate using BeginSend within the callback to finish sending data. So naturally I was very happy to find your blog. :-)

    -John

  4. gaksd says:

    ReaderWriterGate is useless, it has an extremelly hot path. The entire PowerCollection is just an experiment, I’d avoid it at all costs.

    Tried downloading the zip and RAR can handle it but contents are not readable.

    Any fix, just RAR it :)

  5. Obin says:

    Zip is still corrupt.

    Any alternative location?

  6. Greg says:

    Bijan yes that is a possibility

    generally however I would prefer to use a lock free stack :)

    Cheers,

    Greg

  7. Bijan says:

    Greg,

    You may want to consider using ReaderWriterGate Lock (http://msdn.microsoft.com/en-us/magazine/cc163532.aspx) from
    Jeffrey Richter’s Power Threading Library (wintellect.com/PowerThreading.aspx)
    instead of lock inside CheckIn…

    Regards,
    Bijan

  8. Greg says:

    Sean thanks, good catch there was a test for the negative situation but not the positive. Here is the test for the positive situation and the associated code change.

    [Test]
    public void should_return_a_valid_buffer_when_create_memory_is_false_and_memory_is_available()
    {
    //BUG 1273: Buffermanager was not allowing construction when not allowed to create memory
    BufferManager manager = new BufferManager(1, 1000, 1, false);
    ArraySegment buffer = manager.CheckOut();
    Assert.AreEqual(1000, buffer.Count);
    }

    in BufferManager::ctor

    m_AllowedToCreateMemory = true;
    for (int i = 0; i < _InitialSegments; i++)
    {
    CreateNewSegment();
    }
    m_AllowedToCreateMemory = _AllowedToCreateMemory;

  9. Shaun says:

    If you pass in false to _AllowedToCreateMemory but pass an _InitialSegments > 0, you will get a UnableToCreateMemoryException. The test coverage doesn’t seem to cover passing false to _AllowedToCreateMemory

  10. Greg says:

    Ray it will be in public svn shortly, not sure why some have troubles downloading the zip.

  11. Ray Akkanson says:

    The zip file appears to be corrupted or incomplete.

    Ray Akkanson

  12. Joe R. says:

    I noticed that you can change the size; I am interested in finding out if the default size causes the problems I am thinking of. I’ve seen it a lot from wire traces, usually from apps that wrote complex buffer management schemes. Like I said I’ll test it out, it’s likely that there is no problem. If there is a problem, tweaking the defaults or setting up some usage guidelines might improve throughput. It’s a win-win.

  13. Greg says:

    Joe see comment (in code) …. all my stuff allows you to give it a buffer manager so you can determine your own sizes … I just kinda put that there as some sort of sane default for the uninitiated

    Also my stuff doesn’t send 1 buffer at a time it uses the BeginSend(ArraySegment<byte>) overload with N buffers which gets turned into the WSA overload with N buffers internally … 1k is a good default so people can easily control their buffer sizes (i.e. 1k->20k etc)

  14. Joe R. says:

    Have you tested connections which involve real latency? I briefly looked through the code and it looks like the default buffer segment size is 1k.

    It’s possible to mess up the thoughput on a TCP connection to a Microsoft client by buffering to programmer-friendly sizes. MS clients normally ack every-other MSS (not every other packet) so if you run out of buffer space and your bytes-in-flight don’t match up with the MSSx2 you can introduce throughput problems. (Particularly if your outstanding buffer is less than the advertized TCP window size).

    You might take a performance hit using strange buffer sizes, but artificially slowing your throughput (in some cases pathalogically) will use up even more memory;

    I’m very interested in seeing if you avoided this problem. It’s easy to miss, it usually only shows up if the conection has some latency involved. I’ll try to test it out for myself, but see if you notice the the telltale sign of one smaller packet (on the wire) every six or seven packets in a large message.

  15. karl says:

    There’s a known bug for downloads through IE on codebetter. should get someone to look into it. I’ve always just hosted files in an alternative location

  16. Greg says:

    Oran I should actually reflect their code before making that statement to insure my beliefs of it are correct.

  17. Oran says:

    Hmmm, it worked fine when I downloaded the zip with Firefox, apparently IE is messing it up.

    Thanks for the explanation. When you say it “can be used with much smaller messages as well”, do you mean “without unnecessary heap fragmentation due to pinning?” or is there more to it?

  18. Greg says:

    Weird I just downloaded it and it worked fine … I will try uploading it again in the morning.

    It is the same concept (full explanation as to why is in the post from a long time ago) … The difference being it works with ArraySegment out of a big buffer .. and can be used with much smaller messages as well … Also the BufferPool/BufferPoolStream are built on top of it to make things a bit easier to deal with.

  19. Oran says:

    The zip file appears to be corrupted or incomplete.

    How does this compare to the BufferManager in WCF?
    http://kennyw.com/indigo/51

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>