I’ve finally solved this one – not that the resolution makes
me happy, but it’s nice to finally explain the behavior.
I ran into a funny behavior four years ago with WebService
calls on the .Net platform.
At that
time, my idea was to launch webservice calls simultaneously against multiple
backend webservices.
The effect would be
the pipelining of these requests.
To illustrate, if I make a total of ten 5-second requests
sequentially, the total run time from my client is ~50 seconds.
If I make a total of ten 5-second pipelined
requests,
my client runtime is reduced to
~5 seconds.
In my time estimates I’m
factoring out call overhead, but the pipelined requests are potentially more
efficient because the IP transmission overhead is overlaid as opposed to being
sequentially blocked.
While I could launch these from multiple worker threads, the
existence of the Begin… End… functions is far more compelling.
This is because the webservice calls are
fully blocking to the client application and really wastes a perfectly good
thread.
I’d much rather launch the
functions and hold onto a synchronizing object to tell when the processing is
complete.
With the Async methods, I don’t
need to synchronize worker threads and buy into that complexity. It sounds good…
After running into this same fundamental problem again
(different calling behavior, but same pipelining idea), the behavior is
different than I expected based on the documentation. I decided to do some more
digging and resolve this once and for all.
To illustrate, let’s take a simple WebService function
exposed on a service called SimpleService:
public struct Bar
{
public
int i;
}
[WebMethod]
public int
Foo( Bar bar )
{
System.Threading.Thread.Sleep(2000);
return
bar.i;
}
Next, let’s take some client code to make everything clear:
// this class
hold the details needed to get results back
// which
include the web service instance and the IAsyncResult
public class AsyncDetails
{
public
SimpleProcess Function;
public
IAsyncResult AsyncResult;
public
AsyncDetails( SimpleProcess function, IAsyncResult result )
{
Function = function;
AsyncResult = result;
}
}
public void TestRun()
{
ArrayList results = new ArrayList();
SimpleProcess process = new SimpleProcess();
// note -
the behavior only becomes obvious when passing
// a
struct or class - value parameters only help mask the
// real
underlying behavior.
Bar bar = new
Bar();
// Launch
20 running instances and aggregate them in an
// ArrayList
for further processing
for(
int x = 0; x < 20; x++ )
{
// this works the same whether I use a shared webservice instance or
// create a new instance for each call
bar.i = x;
results.Add( new AsyncDetails( process, process.BeginFoo(bar, null, null)));
}
// next,
go through each result and complete processing.
//
real-world app would be aggregating results at this point
foreach(
AsyncDetails detail in results )
{
// assuming that processing is order dependent...
if(
!detail.AsyncResult.IsCompleted )
detail.AsyncResult.AsyncWaitHandle.WaitOne();
Console.WriteLine(
detail.Function.EndFoo( detail.AsyncResult ));
}
}
So what would you expect the output to be?
{0,1,2,3,4,5,6,…,19} in roughly 2 seconds?
Wrong.
The answer is: {19,
19, 19, 19, 19,…, 19} in roughly 10 seconds!
Why is this? It turns
out my operating assumption is incorrect.
My belief that BeginFoo(…) actually launched a request is completely
invalid - it only queues the request.
If they were run as I expected, my Bar struct would be
serialized during my call to BeginFoo(…). Because it’s being queued, the actual
physical call being made to Foo is using the shared Bar struct which has a
value of 19 when it’s actually made!
Furthermore, when you watch this run in NUnit you’ll see that it’s
actually making two requests at a time (two results appear simultaneously, two
second wait, two more results appear simultaneously, two second wait…, and so
on).
My answer to this problem four years ago was to place
synchronous requests in a worker threadpool and manage the threads – it turns
out that was the right call. After
looking at the behavior of .Net, it appears that they are also running these calls
on background threads. When doing this,
you have to be conscientious in the number of threads being used to service
these requests (especially if you’re making these from within a web
application or webservice).
I would have hoped the
.Net implementation would have been more elegant than my thrown-together threadpool, but it turns out that’s just not the case. Perhaps they could have used Overlapped I/O with sockets available since winsock 2??? I suppose if you are making these calls over the public internet you risk being mistaken as an attemped DOS source
[Update 7/20/2006 - It turns out the two connection limit is a machine.config level setting. Check out my followup to this post for more details. While this is configurable, all calls are still being made within .Net's threadpool - so if you are making web service calls from a server-side web application/service, you are dealing with a limited resource.]
[Note:]
The MSDN documentation errantly states: The client instructs the Begin method to start processing the service call, but return immediately. It's the "start processing the service call" line that should be reworded to "queue the service call".