More on Pinning [Perfmon Counters Broken]

In my last 2 posts we spent a bit of time looking at asynchronous
socket buffer management and Continued. In dealing with these two posts I came across a few
things, which appear to be problems in the framework.

 

I promised that I would be a good boy blog more so here we go … The first problem I ran into is the # of pinned objects
counter that the CLR exposes.

Using the code provided in the second post with 500 clients,
run the server (release mode). Bring up perfmon and point it at the server for
“# of pinned objects” located in CLR memory. It should show 7 pinned objects.

 

 

 

Now break the server and load SOS. Run !GCHandles

 

!GCHandles

GC Handle Statistics:

Strong Handles: 28

Pinned Handles: 7

Async Pinned Handles: 0

Ref Count Handles: 0

Weak Long Handles: 46

Weak Short Handles: 9

Other Handles: 0

Statistics:

      MT    Count    TotalSize Class Name

790f7698        1           12 System.Object

79153fd4        1           16 System.Threading.RegisteredWaitHandle

791540c8        1           20 System.Threading._ThreadPoolWaitOrTimerCallback

791138d0        1           24 System.Threading.ManualResetEvent

790f90cc        1           28 System.SharedStatics

790f8688        1           72 System.ExecutionEngineException

790f85e4        1           72 System.StackOverflowException

790f8540        1           72 System.OutOfMemoryException

790f92b8        1          100 System.AppDomain

790f8c24        2          112 System.Threading.Thread

790fa154        5          120 System.Reflection.Assembly

7a77273c        4          144 System.Net.Logging+NclTraceSource

790f8790        2          144 System.Threading.ThreadAbortException

7a772810        4          160 System.Diagnostics.SourceSwitch

79104da8        4          192 System.Reflection.Module

790fb43c        7          252 System.Security.PermissionSet

790ff76c       46         3496 System.RuntimeType+RuntimeTypeCache

7911eb1c        7        17456 System.Object[]

Total 90 objects

 

So it is correct so far, the 7 handles match our shown 7
pinned handles. Start the client and let all 500 connect (it will tell you when
they are done). Watch perfmon until you see a pattern like in the picture here
(note it goes up and then falls back down).

 

 

 

When it has come back down to 0
(will take about 45 seconds) break into SOS again.
 

 

!GCHandles

PDB symbol for mscorwks.dll
not loaded

GC Handle Statistics:

Strong Handles: 27

Pinned Handles: 7

Async Pinned Handles: 501

Ref Count Handles: 0

Weak Long Handles: 3

Weak Short Handles: 11

Other Handles: 0

Statistics:

      MT    Count    TotalSize Class Name

790f7698        1           12 System.Object

790f90cc        1           28 System.SharedStatics

790f8688        1           72 System.ExecutionEngineException

790f85e4        1           72 System.StackOverflowException

790f8540        1           72 System.OutOfMemoryException

790f92b8        1          100 System.AppDomain

790fa154        5          120 System.Reflection.Assembly

7a77273c        4          144 System.Net.Logging+NclTraceSource

790f8790        2          144 System.Threading.ThreadAbortException

7a772810        4          160 System.Diagnostics.SourceSwitch

79104da8        4          192 System.Reflection.Module

790ff76c        3          228 System.RuntimeType+RuntimeTypeCache

790fb43c        7          252 System.Security.PermissionSet

790f8c24        6          336 System.Threading.Thread

7911eb1c        7        17456 System.Object[]

79153be0      501        34068 System.Threading.OverlappedData

Total
549 objects

 

The perf counter is telling us there are 0 pinned objects
but this not seem to agree with SOS. This is very disconcerting since one of
the things we should be watching during debugging is our # of pinned objects to
help watch out for heap fragmentation but it gets worse.

 

This counter only shows us the GCHandle generated pinned
objects, these are not the only pinned objects. The Marshaller also pins
objects when it handles calls to unmanaged code (like when we call into say the
unmanaged socket libraries). In this particular example there are about 3-4
times as many pinned objects than we can see. You will notice that the pins
here are on OverlappedData objects, my actual buffers are also pinned but we
can’t see that they are pinned from here. In doing some research there is
apparently no way to view the objects that the marshaller has pinned so in
order to gauge what’s going on we are bound to only looking at resulting heap
fragmentation and trying to minimize it.

 

This can be seen in
trying to compare the two versions of the server (with and without
buffer management). If you run a !GCHandles where I ran the heap
commands in the previous post you will notice that the same number of
GCHandles are listed for each version yet the buffermanager version has
far less heap fragmentation. This is because the pins on the actual
byte arrays are done by the marshaller as such we can’t see them but
can only view their effect of causing heap fragmentation. There are
other things which are being pinned in this process which is why both
versions show heap fragmentation (3.5 includes a new version of this
which pins far less objects, perhaps we should give it a whirl in
another post?)

 

 

 

I have placed these issues in MS
connect feel free to vote or comment