Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

Asynchronous WebService calls – the truth behind the Begin… End… functions

I’ve finally solved this one – not that the resolution makes
me happy, but it’s nice to finally explain the behavior.

I ran into a funny behavior four years ago with WebService
calls on the .Net platform.  At that
time, my idea was to launch webservice calls simultaneously against multiple
backend webservices.  The effect would be
the pipelining of these requests.

To illustrate, if I make a total of ten 5-second requests
sequentially, the total run time from my client is ~50 seconds.  If I make a total of ten 5-second pipelined
requests,  my client runtime is reduced to
~5 seconds.  In my time estimates I’m
factoring out call overhead, but the pipelined requests are potentially more
efficient because the IP transmission overhead is overlaid as opposed to being
sequentially blocked.

While I could launch these from multiple worker threads, the
existence of the Begin… End… functions is far more compelling.  This is because the webservice calls are
fully blocking to the client application and really wastes a perfectly good
thread.  I’d much rather launch the
functions and hold onto a synchronizing object to tell when the processing is
complete.  With the Async methods, I don’t
need to synchronize worker threads and buy into that complexity. It sounds good…

After running into this same fundamental problem again
(different calling behavior, but same pipelining idea), the behavior is
different than I expected based on the documentation. I decided to do some more
digging and resolve this once and for all.

To illustrate, let’s take a simple WebService function
exposed on a service called SimpleService:

        public struct Bar
        {
            public
int i;
        }

        [WebMethod]
        public int
Foo( Bar bar )
        {
           
System.Threading.Thread.Sleep(2000);
            return
bar.i;
        }

 

Next, let’s take some client code to make everything clear:

 

        // this class
hold the details needed to get results back

       
// which
include the web service instance and the IAsyncResult

        public class AsyncDetails
        {
            public
SimpleProcess Function;
            public
IAsyncResult AsyncResult; 

            public
AsyncDetails( SimpleProcess function, IAsyncResult result )
            {
                Function = function;
                AsyncResult = result;
           
        }

 
        public void TestRun()
        {
            ArrayList results = new ArrayList();
            SimpleProcess process = new SimpleProcess();
 

            // note –
the behavior only becomes obvious when passing
           
// a
struct or class – value parameters only help mask the
           
// real
underlying behavior.

            Bar bar = new
Bar();

            // Launch
20 running instances and aggregate them in an
           
// ArrayList
for further processing

            for(
int x = 0; x < 20; x++ )
            {

                // this works the same whether I use a shared webservice instance or
                // create a new instance for each call

                bar.i = x;
                results.Add( new AsyncDetails( process, process.BeginFoo(bar, null, null)));
           
            // next,
go through each result and complete processing.
           
//
real-world app would be aggregating results at this point
           
foreach(
AsyncDetails detail in results )
            {
                // assuming that processing is order dependent…

                if(
!detail.AsyncResult.IsCompleted )
                   
detail.AsyncResult.AsyncWaitHandle.WaitOne(); 
                Console.WriteLine(
detail.Function.EndFoo( detail.AsyncResult ));
            }

        }

So what would you expect the output to be?

{0,1,2,3,4,5,6,…,19} in roughly 2 seconds?

 

Wrong.

 The answer is:  {19,
19, 19, 19, 19,…, 19} in roughly 10 seconds!

 

Why is this?  It turns
out my operating assumption is incorrect. 
My belief that BeginFoo(…) actually launched a request is completely
invalid – it only queues the request.

If they were run as I expected, my Bar struct would be
serialized during my call to BeginFoo(…). Because it’s being queued, the actual
physical call being made to Foo is using the shared Bar struct which has a
value of 19 when it’s actually made! 
Furthermore, when you watch this run in NUnit you’ll see that it’s
actually making two requests at a time (two results appear simultaneously, two
second wait, two more results appear simultaneously, two second wait…, and so
on).

My answer to this problem four years ago was to place
synchronous requests in a worker threadpool and manage the threads – it turns
out that was the right call.  After
looking at the behavior of .Net, it appears that they are also running these calls
on background threads.  When doing this,
you have to be conscientious in the number of threads being used to service
these requests (especially if you’re making these from within a web
application or webservice). 

I would have hoped the
.Net implementation would have been more elegant than my thrown-together threadpool, but it turns out that’s just not the case. Perhaps they could have used Overlapped I/O with sockets available since winsock 2???  I suppose if you are making these calls over the public internet you risk being mistaken as an attemped DOS source

[Update 7/20/2006 – It turns out the two connection limit is a machine.config level setting.  Check out my followup to this post for more details. While this is configurable, all calls are still being made within .Net’s threadpool – so if you are making web service calls from a server-side web application/service, you are dealing with a limited resource.]


[Note:]

The MSDN documentation errantly states: The client instructs the Begin method to start processing the service call, but return immediately.  It’s the “start processing the service call” line that should be reworded to “queue the service call”.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.

7 Responses to Asynchronous WebService calls – the truth behind the Begin… End… functions

  1. shebert says:

    John,

    It’s really just impromptu test code – I switched to WaitOne when the WaitAll was taking longer to process than expected. I agree, a final solution shouldn’t rely on the order of processing – changes to the framework could change operation.

  2. Steve,

    Out of curiosity: Are you running your test with all services being called on the same server? If so, you might also be aware (or want to be aware) of a different issue altogether that also affects this: The HTTP stack in .NET (even when used indirectly through a webservice proxy) will limit concurrent connections to a single server to 2 to comply with some recomendations in the HTTP spec. While this is useful for browsers, it can really hose you when doing a bunch of webservice calls.

    Fortunately, you can change that behavior. See http://blogs.msdn.com/tess/archive/2006/02/23/537681.aspx for a detailed explanation of it all (with far more detail than you might want to care about, btw)

  3. Mark Brackett says:

    I must be missing something here….Since bar is a struct, wouldn’t SimpleProcess.BeginFoo get it’s own copied bar anyway? Or is bar, since it’s defined server side as a struct, being defined by the WS proxy as a class on the client?

    The 2 requests at a time is an artifact of the HTTP 1.1 (RFC2068) specs. Section 8.1.4 states:
    “Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD maintain AT MOST 2 connections with any server or proxy.”

    There’s a regedit http://www.winguides.com/registry/display.php/536 that’ll increase it in IE, but I don’t know if .NET would pick up on the change.

    Note that I haven’t looked at the actual implementation of Begin/End, but I think using the ThreadPool is a reasonable tactic and assumption for the Begin/End requests. Your tight for loop, your lack of locking on the bar.i read/write, and the 2 request limit I think may be artificial. It’s not like the call isn’t actually made until you call EndRequest (which would be pointless), it’s just that you can count to 19 and change a property faster than the worker thread can open a socket and negotiate the HTTP request.

  4. secretGeek says:

    >two results appear simultaneously, two second wait,
    >two more results appear simultaneously, two second wait…,
    >and so on).

    could this be because IIS is limiting the number of connections per ip address down to two, as is the default behaviour? what if you added this to your web config?

    <system.net>
    <connectionManagement>
    <add address=”*” maxconnection=”100″/>
    </connectionManagement>
    </system.net>

  5. johnwood says:

    I don’t think you can assume that the threads will be executed in the order they were queued. That’s dependent on the context switching and the relationship between the managed and unmanaged threads, it’s certainly not something you should depend on. Why don’t you do a .WaitAll or .WaitAny instead?

  6. Interesting, and a further validation of my latest approach towards calling webservices asynchronously in 1.1: I use Juval Lowy’s BackgroundWorker Component for 1.1 and call the webservice from the DoWork event handler. This vastly simplifies things and makes it understandable for junior developers as well.

  7. Jiho Han says:

    I don’t understand why “bar” is shared. If it’s a struct, wouldn’t each call to Foo get a copy? So even if it does take 10 seconds altogether, I’d think you should receive {0, 1, 2, 3, … 19}. Am I missing something?

Leave a Reply