Building polyglot packages for OpenWrap and NuPack

As we approach a first full-featured OpenWrap release, it’s time to provide package builders some guidance in how to build good packages.

For a week now, the OpenWrap client has supported the NuPack package and feed format for assemblies. We don’t support powershell scripts so far, as OpenWrap doesn’t have a dependency on it, but assemblies in the /lib folder are fully supported.

Test your packages

The easiest way to test a nupack package is to use the convert-nupack command available in OpenWrap.

PS C:\tmp> o convert-nupack .\Castle.Core.1.1.0.nupkg.zip Castle.Core.1.1.0.wrap
# OpenWrap v1.0.0.0 ['C:\Users\sebastien.lambla\AppData\Local\OpenWrap\wraps\_cache\openwrap-1.0.0.18826071\bin-net35\OpenWrap.dll'] Package successfully converted.

This will convert the NuPack into an OpenWrap package. You can then add this package to your system repository or to any other project as usual.

Understand framework profiles

The resolution logic between NuPack and OpenWrap is slightly different.

NuPack will add references to the dlls it finds in a folder called /lib/[Framework][Version], and only in that package.

OpenWrap, because it focuses on solving dependency resolution at run-time too, supports dynamically swapping assemblies by name, and will process any compatible assembly in the package. An example will make it clearer.

The Newtonsoft.Json package contains a .net 2.0 specific dll in /lib/20 called Newtonsoft.Json.Net20.dll, and a 3.5 version in /lib/35. Only one of those references should be included.

NuPack will only add the /20 or the /35 libraries to your project. OpenWrap however will find two assemblies named differently, and provided you are resolving those assemblies in a .net 3.5 environment, will import both.

The correct way to package your assembly is to name the same assembly targeting different frameworks the same, and put them in the correct folder.

To sum up, here’s the structure that should be used to package your assemblies targeting different versions.

NuPack

  • /lib
    • /20
      • Newtonsoft.Json.dll (formerly /lib/20/Newtonsoft.Json.Net20.dll)
    • /35
      • Newtonsoft.Json.dll (formerly /lib/35/Newtonsoft.Json.dll)

OpenWrap

  • /bin-net20
    • Newtonsoft.Json.dll (formerly /lib/20/Newtonsoft.Json.Net20.dll)
  • /bin-net35
    • Newtonsoft.Json.dll (formerly /lib/35/Newtonsoft.Json.dll)

Version your packages

Versioning of packages is vital for good package management. I’m not entirely sure what the versioning behaviour for NuPack is, but OpenWrap is quite opiniated. Each package can and should be versioned using the Major.Minor.Build.Revision format.

When you take a dependency on a package version however, OpenWrap will ignore the Revision component, and automatically import any new revision of a package next time you update.

The reason for this is to enable hot patching packages to correct issues without users having to change the dependency they declared.

Let’s take an example. Let’s push a package to a remote repository.

PS C:\tmp> o publish-wrap server .\castle.core-2.5.1.0.wrap -debug
# OpenWrap v1.0.0.0 ['C:\Users\sebastien.lambla\AppData\Local\OpenWrap\wraps\_cache\openwrap-1.0.0.18839165\bin-net35\OpenWrap.dll'] Publishing package 'castle.core-2.5.1.0.wrap' to 'server'

I now add a dependency to my project (you would do the same using xml in your .nuspec if you build for NuPack).

depends: castle.core = 2.5.1.0

And I ask OpenWrap to fetch the latest packages.

PS C:\tmp\castle.user> o update-wrap
# OpenWrap v1.0.0.0 ['C:\tmp\castle.user\wraps\_cache\openwrap-1.0.0.18839165\bin-net35\OpenWrap.dll'] 'Project repository' up-to-date as 'openwrap-1.0.0.18839165' <= 'openwrap-1.0.0.18839165'. 'System repository' up-to-date as 'Castle.Core-2.5.1' <= 'Castle.Core-2.5.1'. Copying 'Castle.Core-2.5.1.0' from 'server' to 'Project repository' 'Project repository' up-to-date as 'openfilesystem-1.0.0.18676829' <= 'openfilesystem-1.0.0.18676829'. 'Project repository' up-to-date as 'sharpziplib-0.85.5.452' <= 'sharpziplib-0.85.5.452'. Making sure the cache is up-to-date...

Of course, you would’ve had the same result by simply asking OpenWrap to add the pacakge with the add-wrap command.

If I now push a new version of castle.core, say 2.5.1.1, and execute an update, the package will be automatically updated to the latest version, no question asked.

To recap:

  • Do change the revision part to deploy hot fixes to your package so people don’t have to update their dependency files
  • Do change the build, minor or major components when you want to let people depend on a different version of your API.
    • Update the build component when you add new classes or method
    • Update the minor component when you rename, remove or change the signature on your API.
    • Update the major as you wish.

Don’t sign your assemblies

One of the features OpenWrap provides is dynamic swapping of assemblies based on your framework version and platform (x86, MSIL, etc). This can only work if you don’t depend on signed assemblies, and if you don’t sign yours.

If you have signed assemblies in your workflow, you will have to create publisher policy redirections, a lot of xml that goes in your .config files. We may automate this process in the future, but if you’re going to automatically add those entries to enable new versions of an assembly to be used, you may as well Keep it Simple Stupid and not sign them.

There is no reason to use assembly versioning when package versioning is in place. Contrary to what some may think, it doesn’t provide identification of assemblies (anyone can generate a key), it doesn’t protect your assembly from decompilation, it is only used for ensuring that software compiled against a certain version of an assembly doesn’t get updated without a recompile, which is exactly what we’re trying to avoid.

Understand your dependencies

If you follow the versioning model I mentioned above, you will know which range of versions your assemblies depend on.

Problem is, at a very minimum, it’s unlikely that version 3 of NHibernate will have exactly the same API surface as version 2. Don’t take an unversioned dependency!

Always try to provide at the very least a Major.Minor dependency in your descriptor. If you’re worried about the future and people being broken, this is exactly the reason for the Revision auto-update behaviour we’ve discussed earlier.

If a new version of a package comes out and doesn’t break your package, you can simply edit the package with a modified revision range, and people will get automatically updated next time they issue an update-wrap command.

Conclusion

Building packages is not an easy thing to do, and supporting multiple uncoordinated package managers is not easy either. If you follow these guidelines, you should however manage to build your packages correctly.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Pingback: | Blog | OpenWrap 2.0–Package locking

  • http://codebetter.com/members/serialseb/default.aspx Sebastien Lambla

    @Kiliman

    Yes, I’m aware of semver, and decided that it was unlikely to be adopted (as it makes no use of the revision number people are used to use in .net). As such, I recommend the same logic shifted by one.

    That said, openwrap supports either formats, as you take a dependency range you define yourself, if your package is semver, all the best. If it’s not, we still support that.

    My recommendation is still the one above until such a time semver gains more adoption.

    As for packagers doing what they do today, this is exactly why I issued this recommendation, so people know what is expected of a package manager.

    I fully expect at some point that the role of packager will become one of deciding what to release and when, separate mostly from developers, so that packages can be compiled against the right versions and with the right version ranges, something developers don’t always think about.

  • http://blogs.systemex.net/Kiliman/ Kiliman

    For the most part, I believe package maintainers are using the same version numbers that the project owners have assigned to their assemblies. The filenames are also defined by the project owners. These 2 groups are not neccessarily the same.

    Since package management is a (relatively) new thing for .NET, I’m sure it will take a while for people to migrate their build process to create binaries more suitable for packaging.

    Currently, I believe that package maintainers are simply taking the binary distribution for the project and building a package around it. Except for creating a /tools and /lib/[FrameworkVersion] folder, they are simply copying the bits as is.

    This is how Nu pretty much operated for creating gems. We defined conventions on the best ways to layout your project, and we worked with project owners/package maintainers to help drive adoption. However, we didn’t want to be draconian and say this is how it has to be. Ultimately it was up to the project owner to choose a layout that works best for him.

    As for versioning, I agree that it’s a mess. Although Microsoft has a well-defined version number scheme, it doesn’t appear that all projects are following it. I’ve seen version 1.4 that is not compatible with version 1.2. Add signing to the mix and you’re back in Assembly Hell.

    With Nu, we started promoting Semantic Versioning (semver.org). This basically follows the .NET versioning model, but actually gives strict meaning to each part: Major.Minor.Patch.

    Patch version is incremented for COMPATIBLE bug fixes. Minor version is incremented for COMPATIBLE new features. And Major version is incremented for INCOMPATIBLE breaking changes.

    This means that you should always be able to pull down the latest 1.x version and know it won’t break your project. However, this isn’t a problem that the project maintainers can solve. This requires each project owner to correctly version their code. As long as they’re creating assemblies where 1.4 is not compatible with 1.2, then we’ll never be able to solve this problem.

    I notice that OpenWrap is using a different version model. I recommend that you look into SemVer and incorporate that into OpenWrap. SemVer is already used by Ruby Gems, so it looks like it can become an industry standard.

    Sorry for the long comment. Thanks for continuing to push the envelope on package management. I believe having different options will ultimately result in better solutions for all.

  • http://codebetter.com/members/serialseb/default.aspx Sebastien Lambla

    @Ian this should be fixed now, let me know if there’s still an issue.

  • http://codebetter.com/members/serialseb/default.aspx Sebastien Lambla

    @Krzysztof: Yes, that’s a problem indeed. It’s a choice that is up to end developers at the end of the day. Maybe if signing is that important it should be done at build time by resigning existing assemblies with a new key, and maybe we can help with that.
    The other option is to simply provide two packages, one signed, one not signed, using a naming convention, or the upcoming namespace support for package names. It’s then up to users to choose which package they want, and for them to deal with the pain their policies are adding.

    @Paul: I’m not sure how far we’ll go in any direction. We’re adding a warning at build time to let people know that if they sign their assemblies, all hell will break loose for them when something changes.
    We’ll also add that warning whenever a package is pulled down that contains assemblies that are strongly-named, either through an update or an add.

    Now we either by default prevent signing assemblies and strip out assembly strong-naming, and provide a -allowsigned parameter that you need to add to preserve those, or we do the opposite, warn and provide an optional -unsign to strip them out automatically.
    I’m tending towards the latter myself. This won’t be part of the first release, but as we’re going to do a weekly release cycle, we’ll see what we do based on feedback.

  • http://thesoftwaresimpleton.blogspot.com Paul Cowan

    Waiting for the .NET world to catch on that signing assemblies should NOT be the default will take forever if it ever happens.

    I take it OR can unsign everything for you?

  • Ian Horwill

    The code snippets appear blank in Google Reader. I think the background colour is ignored so you get white on white.

  • http://kozmic.pl Krzysztof Koźmic

    The issue with not signing assemblies is not that obvious one. Some companies have a policy to sign everything and if you want to let them use your project you have to sign as well.

    Anyway – welcome to CodeBetter!