The .Net community and the negative view on assembly references

Why is it that we abhor large numbers of references in our .Net projects? Beyond the obvious that is (I don’t care about start up time, or any other performance hogwash). It seems that early in my career I thought 10 assembly references was absurd, now that number is MUCH higher (almost without limit) but this view is intensely not the view held by the larger .net community. I take a look at other communities outside of .net and see people doing a much better job of pulling together a multitude of libraries to construct even simple sites versus the vast majority of .net projects I have seen that seem to go out of their way to avoid a dependency hit. Not to mention the enterprise projects that build massive systems and taking no more dependencies than exist in the BCL.

This bias of not wanting to import a larger number of references has impacted the way I design my .net code, and I am not sure I would do the same if I were in another language. For instance, I have a library called Magnum that is a junk drawer of functionality for my other projects. Every now and then someone will see the reference in another project and will go take a look at it. To there surprise, it contains a vast and deep amount of code that could be very useful to them. But to import it means to import a LOT of stuff. Why don’t I break up this codebase? Because of my negative view on assembly references. “Le sigh.” Should I just start splitting things up more? How does this splintering of Magnum effect other things?

A part of me wonders if this isn’t caused by the same effects of ‘The Tale of J. Random Newbie’ in the Art of Unix Programming pg 415. Go read the 4 pages or so. I will wait.

The Art of Unix Programming

It would not surprise me if the lack of OSS experience and transparency in the large part of the .Net community isn’t responsible for this.

Of course, I can’t, in practice, ignore the impacts of additional assembly references on the performance of my applications. Has anyone actually measured this?

Thoughts?

-d

About Dru Sellers

Sr. Software Engineer at Dovetail Software.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.facebook.com/Ahmedn1.Dawod Ahmed Dawod

    I agree with you that writing the code from the start is a very great joy to any professional programmer but let’s say that we don’t want to “reinvent the wheel”
    DLLs are used to save developers’ time and effort to produce more powerful applications

    you can spend your free time enhancing others’ dlls and make better ones

  • stacey

    I am just OCD. That is all. Never said there was valid method to my madness.

  • http://wizardsofsmart.net/ panesofglass

    Such an excellent response! In the F# stuff I’ve been working on, I’ve started thinking of my files as “assemblies.” With the ability to publish files as Content in NuGet packages, it’s possible to start including files as dependencies directly into your code base. No ILMerge needed to get a single, deployable .dll or .exe.

    The question will certainly arise as to why in the world anyone would want to shove a lot of stuff into a single file. Well, it works pretty well in F#, as in a number of dynamic languages. Also, with tools like R#, there’s really no point worrying over file organization when you can easily navigate to any part of your code base. I’m sure others will disagree, but it’s working for me. (Sorry for the derailment there at the end.)

  • http://twitter.com/PhatBoyG Chris Patterson

    Magnum ilmerges and internalizes Newtonsoft, so you should never see it anyway. I’m not sure why that would be a problem for anyone.

  • http://twitter.com/PhatBoyG Chris Patterson

    That’s why I use the Power Tools version, it’s much quicker.

  • http://weblogs.asp.net/bleroy Bertrand Le Roy

    Oh, simple: there is no appdomain or assembly, a library is just a file, so its impact is pretty much proportional to the size of the file. It seems to me like in that case, it would actually be better to have lots of small files that you all use rather than less big files, that you are not going to entirely use.
    As far as Orchard is concerned, I’ve long wanted to have a lightweight plugin system relying on a dynamic language model, for this kind of reason, so that many small extensions have minimal impact (we actually had that on early prototypes).
    Oh, and there is the problem that you can’t unload an assembly, whereas most scripting engines make garbage collecting code easy.

  • RichB

    I know of Magnum. I’ve considered using it. But I don’t – and not exact opposite reason of what you suggest above. It’s too large. 

    Our CI server already produces 200MB of .zips on every successful check-in. Adding magnum would increase this by 2*4*zip(sizeof(magnum)). 

    And I just want to use a small part of Magnum.

  • J Williams

    When compiling, every project in our solution has a basic overhead of 3 seconds. Merge 2 projects into 1 and you knock at least 3 seconds off the build time. We have 170 projects, so by lumping everything into a single project we can save almost 9 minutes - of our 12 minute build time! Separate assemblies are only needed if multiple apps need to share them. OO separation of concerns is handles with class/namespace/module – do we need all this assembly-level modularisation? At the cost of 10% of our development budget?

    Size and load times have similar relationships.

    We obfuscate our app to protect it. Put it all in one massive file and we’d get a bit more security through obscurity. So why do we break up our apps into hundreds of tiny dlls with names like “MyApp.Licensing.dll” or “MyApp.TopSecretAlgorithm.dll” that take the hacker directly to the code they want to reverse engineer?

    But for me the biggie is this: .net References are a complete pain in the backside. Even a reference pointing at a *specific file* results in a probe sequence that reaches out past mars and pluto to see if it can pluck an old incompatible version of the dll out of the ether. If there is a wrong copy of a single dll anywhere on your network, you can guarantee that one day your build, for no obvious reason, will use that dll instead of just using the exact the file you pointed it at. On several occasions our application has crashed and burned when one of our 170 dlls has linked inexplicably to a completely incorrect (missing) version of a referenced dll. We only found this through manually testing every feature for every release. We now build in a “clean” environment where old versions of dlls are not allowed to be present anywhere on our build server’s filesystem, and the problem has not recurred. Why can’t the reference just use the exact filename we gave it?!

  • Anonymous

    Thanks Captain Obvious ;)

    Lenny, when you are going through the thought process, what kinds of things do you like to think about? Or, what does it depend on?

  • http://twitter.com/lennybacon lennybacon
  • Jim Cooper

    Ah, that wan’t clear to me. I don’t have a problem with 3rd party references, as long as they’re under version control. Because I prefer fewer, larger assemblies, I can have quite a lot of those.

  • Jdscolam

    I think we tend to rely on the “solid OO principles” assumption when it comes to keeping ourselves and other devs honest.  Sometimes whether intentionally or accidentilly we break principles, and it seems to me that it would be harder to fix if the principle break was in a giant assembly versus in a smaller dependent assembly.  I guess that’s the point of “componentization”, having lots of small simple solution components, and combining them to create the complex solution.

    Am I off-base here? 

  • Jdscolam

    Wow, this timing couldn’t be better, we are having this exact discussion on the team I’m on.  I tend to prefer to keep my dependencies isolated into separate assemblies.  Partially for reuse (e.g. I know I’ll use this bag of utilities in other projects, best to package it separately), partially for architectural reasons (e.g. my repository/repositories are separated out from the domain, host projects separated from business logic/domain, etc.).  Granted, it seems my thoughts on the architectural side have had some serious holes poked in them by the other comments.  The rest of the team (two intermediate devs, and one other senior dev) are really interested in cutting down our number of projects and have been merging assemblies.  I keep going back and forth on the issue.  It really seems to me that projects are easy ways to organize code, but might that be a code smell?  I could go either way.  What do you guys think?

    Also, the team is using “Developer Productivity” as a “metric” on how easy it is to maintain while developing the project. Definitely interested in further thoughts. 

  • http://jamesmckay.net/ James McKay

    In theory, you would only have NHibernate references in a single project. However, in practice, this isn’t always possible. Hiding NHibernate behind a single abstraction layer means that either (a) you have to limit yourself to a subset of functionality which will prove to be almost useless if you need to do any meaningful performance optimisation (ie avoiding SELECT n+1 problems all over the place) or else (b) you have to make your abstraction layer so complex and specific that it’s another whole level of DRY violations in itself.

    Besides, I’m just using NHibernate as one example. There are a lot of other things that could cause the same problems: logging frameworks and IOC containers, to give just two examples.

  • Mike

    NuGet will change this.

  • Anonymous

    Excellent points! NuGet and OpenWrap are still young but they are helping this problem dramatically. Using a DVCS (or just not a crap VCS) I also think helps a lot. We don’t have a 1st class RVM, but you can choose your framework to run against in .Net. Do we need a bundler? Since we tend to use private copies of binaries rather than a centralized one.

  • Anonymous

    I would disagree that a multitude of assemblies in a solution is a violation of DRY. In the case of upgrading from NH 2 -> NH 3 I would only have to modify one project (the part of my project that relates to persistence). The problems you experienced DO make me think though that there was a violation of DRY. Does that make sense?

    re: the compile time issue, there are solutions for that, but it could very well be a factor of our negative view on assembly refs.

  • Anonymous

    AH, that’s a good one. The dog slow performance of VS Add Ref!

  • Anonymous

    Agreed, but that still doesn’t address the point about why do we have this negative view. There are many ways to skin the cat technically, but what about in the hearts and minds of other developers?

  • Anonymous

    Jimmy Jimmy Jimmy. :) – beyond I/O perf do you have any other reasons? And the GAC is just like a more strict GEMS repo right? “No Soup For You!”

  • Anonymous

    Why do you think that dynamic languages have a lower impact? Or rather why do you think that static languages have a higher impact?

  • Anonymous

    Agreed, some people do silly things. So is this no 3rd party dlls a backlash to this bad behavior for what would otherwise be a good practice – see the bit on the J. Random Newbie – right?

  • Anonymous

    I would agree with that. But it’s a smell rather than a fact. If you run your entire app in one dll but have everything isolated via solid OO principles and clear boundaries. Even still I think most .Net projects try to avoid bringing in a bunch of 3rd party dlls vs other communities.

  • Anonymous

    Specific to my article, I was not talking about project references but about 3rd party refs like NHibernate, Castle, StructureMap, etc

  • Anonymous

    which is why I called it hogwash in the beginning. :) 1. download size (thankfully not an issue for me) 2. reducing dep mgmt complexities (agreed) 3. You don’t have to update, unless you have update OCD 4. Conflicts yup 5. true, you gotta keep your house in order.

  • http://implementationdetails.com Petermtate

    The first problem with external dependencies is managing them.  Tracking, updating and managing the versions of the assemblies. I think this is partly a cultural aspect, package management tooling is just taking off now (with NuGet) while in the Java community it seems like this has been engrained for some time (using Maven repositories for example).

    Secondly, and this may be specific to our company, is the legal issues. Every open source package we want to use has to be run by our legal department to review the license, which is additional friction in using external tools.

    As for performance, the big hit is cold startup time (e.g. after a reboot).  Cold Startup time is (roughly) Warm Start Time + Disk IO so the more dependencies you have the longer the startup time.  If you’re requiring strongly named assemblies, the whole assembly has to be loaded for strong name verification.     In our case cold start time was 25 seconds, warm start time was 5 seconds.  This can be reduced by putting things in the GAC (thereby skipping the need for additional verification), but then the GAC causes other problems if you have multiple versions of the same assembly in use (a new form of DLL hell).

    Interestingly, the people at our company that want to reduce the external dependencies seem to be the same ones who want to break solutions into many tiny projects.  I’m looking at a solution right now with 50 projects, many are very small (in fact one has exactly one class, with no methods, and two constants).  This decomposition has the (noble) goal of matching the physical design  and logical design of the application.  The downside (as Thomas Weller has already mentioned) is that this decomposition results in increased build times, and painful deployment.

  • Anonymous

    Me too, I have the same aversion to numerous dependency, with no real good reason…..
    Although, every time I make a new executable, and I need to add 15 references to be able to use some of my DLLs, the waste of time irks me, particularly when I forgot a few and I should do it again and again…
    But I take solace in the resignated knowledge that complexity can (and will) only grow as project mature, I just have to suck it up!!!  

  • http://crisdelvallelife.blogspot.com/ CoreAn_Crack3rZ

    This all depends to the programmer (I guess, hehehe). 

  • Anonymous

    I think it depends on the nature of the references.  If they are all external libraries, it may not be an issue.  But if you have a lot of project assemblies, this might be a code smell that you are segregating your code too much, and not properly establishing context boundaries.  Bob Martin’s book “Agile Software Development, Principles, Patterns, and Practices” has some great material on how to shape package (assembly) dependencies in code.

  • Jim Cooper

    Lumping all your code in one assembly can be the right thing to do, for certain.

    As someone else said, splitting a solution into many assemblies does not a better architecture make. My experience is that the architecture is generally worse, when projects are used as a code organisation tool. You should only make exes and dlls that make sense from a physical POV. E.g will this all ever be updated by itself, or replaced by a substitute, etc? If the answer is no, then it shouldn’t be a separate assembly.

    There are two problems with too many projects in a solution. One is compile time, and the other is dependency issues.

    I have recently worked on code with both these issues. The solution (not that big in LOC terms), took up to 20 minutes to compile. 

    It also had an issue where the dependency of one assembly on another was the wrong way round, and as a result, there were functions that had 50 or so parameters, and all sorts of other crap. Fixing this one dependency issue we estimated would take about 3 man-months, and run the risk of breaking the application entirely (naturally, this type of code had no unit tests!).

    Had they just been namespaces in the one assembly, then most of the problem code just would’t have happened.

    Moral of the story? Only create executables and DLLs when you actually need them. Using them for code organisational purposes only is a BAD IDEA. That’s what namespaces are for.

    Too many people should create a new folder, and create a new project instead.

  • Jim Cooper

    It’s not a silly reason if you are talking about performance of the compiler. Lots of projects make things compile slower. IME, sometimes very much slower. It’s a complete PITA to wait 20 minutes for a build.

  • Thomas Weller

    If you think of different assemblies that you have to reference, then I agree with you. What I mean is: You have to manage more *reference instances* when having your code distributed across many assemblies. 
    If you divide your code into assembly A and B, then you might to have reference assembly C from both, and you also have to take care that no version mismatch is occuring during runtime. If you instead put all your code into one assembly, you need to reference assembly C only once, and you wouldn’t have to care about versions.
    On the other hand, it’s of course also true that it is a bad thing if I would be forced to satisfy a bunch of useless dependencies, only because a referenced assembly would require this.

    My strong experience is: The less assembly references you have to manage in a project, the better. But this does not mean that there cannot be strong reasons to isolate certain chunks of functionality, especially if you want to share it across projects, as you mentioned.
    What I primarily had in mind – and I’ve seen this much too often – are solutions which have dozens of projects with only a few classes each, and the code will be used exclusively by that solution. This really is a severe architectural smell. – I can remember one situation when I spent two days only to figure out the runtime dependencies on such a solution’s build server…

  • http://jamesmckay.net/ James McKay

    There is a very good reason why too many assemblies in a solution are a bad thing: it is a violation of DRY. If you’ve ever tried to upgrade NHibernate 2 to NHibernate 3 in a solution with thirty projects, you’ll know just how painful it gets. Even worse, NHibernate 2.0 to NHibernate 2.2 where extra assemblies were introduced (e.g. NHibernate.ByteCode.Castle). Then in the same project I found two different copies of the same version of NHibernate in different places were being referenced by different projects, resulting in warnings about conflicting references that were almost impossible to locate.

    I don’t think a lot of assemblies in your project will have much of an impact on runtime performance, but it will certainly have a negative impact on compilation times. The compiler re-loads (and re-copies) all the references for each project in your solution, so for a large project, the time to do this will increase as O(n^2).

  • http://twitter.com/jcdickinson Jonathan C Dickinson

    There is a marginal amount of overhead when loading an assembly (probing, allocating memory, module initializers, etc.) – on top of that the code needs to be JITted (however it would be in one humungous assembly anyway).

    To mitigate the assembly loading overhead you can ILMerge dependant assemblies into a single one – but I think it’s hardly worth it; the overhead is TINY.

  • Matthias

    I’ve been developing .net for 5 years now from the v1.1 on. And the most trouble we had with assembly references is at creating patches for existing customer installations. We use third-party packages for ui and some special operations and every integration of a new release of these packages leads to an immense effort in creating the patches for our delivered solutions or even breakes the upgrade chain (using real patches, not a new full setup).
    As Workaround we’ve started to combine the third-party assemblys from one vendor to one assembly, so that we’ve only one reference to that package in the solution which can be replaced more easily on upgrade needs. This works for some of our third-party elements, unfortunately not for all (Architekture or Licensing causes).
    The next point is the problem behavior with plugins. Some of our Solutions use Plugins which are separated as assemblys. References to a third-party assembly in the Master App and in the Plugin must be always at the same level, which means, we have to update the plugin, even if it runs fine, when updating the references of the main app. Rolling out these updates is also very complicated (ever tried to update a gac entry with a setup patch?)
    I would like to see a version compatibility info direct on the referenced assembly, like “this is v3.4.1.6 and it is backward compatible in interfaces down to version v3.4.0.0″, this would save us a lot of time for reference handling. The current mechanism with machine-, app- and gac.config respective policies for reference remapping is to time-consuming to handle. I thinks thats one point why you mostly see no patches instead of full setups with small version steps.

    Greetings

  • http://twitter.com/dagda1 dagda1

    I wish more people had a problem with too many .dll references. The vast majority of big shops don’t stray too far from the BCL.

    This is definitely a cultural thing and a lack of a decent package manager or a choice of package manager is part of the problem.  Ruby gems, rvm and bundler make importing gems almost too easy.  This does not mean you don’t still have problems and the high abstraction means when it does go wrong, you can feel pretty lost but it is a great system.

    We’ve still some way to go in .NET and I personally use openwrap.  The use of a good DVCS like git is important.  As branching is cheap, we can easily branch off and run some spikes without much disruption.

    Sadly few of us in .NET use git or mercurial and TFS is unlikely to get its act together or go away.

  • Grant Walker

    I’m not sure that I agree with your assertion that fewer assembly references will automatically result in fewer dependencies.  By packaging a “huge” amount of code in one assembly (in multiple namespaces naturally) one could end up with an assembly which has dependencies on a large number of other assemblies.  If the consumer of your assembly only wanted to use a small cross-section of the full functionality of your assembly, they would be forced to satisfy ALL of its dependencies.

    I do not doubt the affect that dependencies on differing versions can have on this topic; however I don’t believe that “lumping” all your code into one assembly (especially if you have the intention of sharing it) is the right thing to do (architecturally or otherwise).

  • http://thomasweller.pip.verisignlabs.com/ Thomas Weller

    Personally, I feel that the number of assembly references has a massive impact on application loading times – especially in Web applications. But this is ‘only’ gut feeling, I also haven’t seen concrete, measures on this yet.

    But the most important point to me for minimizing assembly references to the bare minimum is just architectural cleanliness! Mainly for two reasons:
    1.
    In large projects, you have a lot of third-party references anyway. This in turn leads to cross-dependencies. It’s true that we don’t have the ‘dll hell’ anymore, but we still have some kind of ‘assemblyversion limbo’: We need to make sure that all used assemblies reference the desired version of other assemblies that we have in our project. My favorite example for this is NHibernate, which internally uses log4net. If you also use log4net otherwise for your application’s logging  stuff, then you have to make sure that your log4net version matches the one required by the NH assembly, either through natively using it or through having the necessary version redirect in your configuration file. This alone would be simple, but as it is with big projects, the number of dependencies tends to explode…
    2.
    It is much better to package a huge amount of code into one assembly which is internally structured by namespaces, than arranging all this code into a larger number of smaller assemblies. This has many reasons – you can refer to http://codebetter.com/patricksmacchia/2011/05/30/on-partitionning-net-code/ if you want to read more about it. The main reason for me is: it raises the quality of a project’s architectural design and code quality in general…

    So, thinking twice before adding another assembly reference to a project is a matter of quality to me – and therefore a matter of doing a good job. It fights two major enemies of software development at once: Complexity in general and dependencies in particular.

    And no, I don’t really care if others are doing it differently. This does not necessarily mean that they are doing it better or have better reasons…

  • http://twitter.com/doobiaus Craig D

    Personally I don’t really concern myself with it, I have no need to at the moment. 

    My thought though, is that if it is an issue, you could look at loading the assemblies  dynamically using something like MEF.

  • Anonymous

    I have hit problems with large numbers of projects and files getting copied around, but I haven’t measured it for dependencies. Luckily, I just put everything in the GAC and all my problems are solved.

  • http://weblogs.asp.net/bleroy Bertrand Le Roy

    > Has anyone actually measured this?
    Yes, in Orchard we have. It is not negligible, which is a bummer in a highly modular system. Then again, it is not negligible *because* we are highly modular, and people tend to add lots of stuff. I agree with Ayende that in ordinary apps it does not matter and is just silly. I think it’s not just the OSS culture that is lacking though, it’s also that other environments are mostly dynamic languages where the impact is lower, or at least of a different nature. I think.

  • http://nikosbaxevanis.com/bonus-bits/ Nikos Baxevanis

    Most of “architects” tend to break the solution to a huge number of assemblies believing that way they achieved a “layered architecture”.. in fact the code most of the times is tightly coupled, there is no SoC and at the end all these assemblies are loaded into the same AppDomain.. :$

  • Damien Guard

    To me a large number of dependencies within a single project indicate that the project itself doesn’t have clear boundaries.

    Very much like too many usings indicates a likely violation of a Single Responsibility Principle at class level.

    [)amien

  • Ayende Rahien

    Perf is a very silly reason to do that.
    It is _very_ unlikely that you’ll be able to get a meaningful difference on modern hardware in any non contrived test.

    Reasons to want to do that:
    * Reducing download size
    * Reducing dependency management complexities
    ** If I am using 5 libraries, I might need to update one of them every other month. If I am using 25, I might be needing to update one of them every week.
    * Reducing the chance of conflicts.
    ** NHibernate & Castle were a great example of a problematic dependency cycle, until we broke that.

    And there is the old:
    “Look, no reference to NHibernate in the model DLL”!!!

  • http://blog.abodit.com/ Ian Mercer

    Specifically in Magnum the tie to an entirely different open source project (Newtonsoft.Json) is an issue for me.  There’s some great stuff in there but I didn’t necessarily want that version of Newtonsoft in my solution.  Magnum would be better as maybe three different DLLs, one without a web dependency, one with, and one with just the Newtonsoft dependent bits in it.