OpenWrap 2.0–Package locking

Well, this one was scheduled for much later in the development process of OpenWrap, but ended up being built last week.

Why locking, a historical perspective

OpenWrap is very much a http://semver.org-friendly system. When you take dependencies on packages, we ignore the revision number (that’s the 4 in 1.2.3.4). The reason is two-fold: the first one is that we believe in semver.org. The second one is that the revision is supposed to be used by library developers that want to hotfix a package urgently for security reasons, a side-effect of us ignoring that number alltogether.

The point is, in your package version, you must not deploy incompatible changes using revision numbers. It’s been like that since the inception of OpenWrap, so the behavior we’re talking about has been present in public releases since May 2010.

Many months later, nuget was announced, and people started publishing packages with the freedom to do whatever they want with their version numbers, even if it’s following no convention at all, rendering version numbering completely useless. It follows the nuget mantra of we’re not changing what you currently do, and is at odds with the OpenWrap mantra of we make it very very hard for you to shoot yourself in the foot.

As it happens, guidance on that issue was published nearly a year ago at http://codebetter.com/sebastienlambla/2010/10/15/building-polyglot-packages-for-openwrap-and-nupack/.

An OpenWrap customer recently encountered an issue where a package was pushing visible changes in the revision number, and that ended up auto-updating their solution, breaking their code. Definitely not something I want OpenWrap users to deal with.

At this stage, there are two ways to solve the problem. The first one is to do whatever the other package manager does, even if it leads people in the pit of failure. The other one is to add functionality that solves the problem in a way that puts people in the pit of success, while still preventing people from doing stupid things. That, and asking nicely the author of the package to reconsider their versioning scheme (as a side-note, the problem already happened with nhibernate, they refused to change anything because of one tool, while raven changed the versioning scheme within two hours, kudos to @ayende for being on the ball).

Introduction

It is often the case that you want to make sure everyone in your team uses the same package versions. You could of course ask everyone to not update anything, but if someone adds a new package, dependencies between an old and the new package could be impacted by the update.

Or maybe it is the case that at development time, you want to freeze in time certain packages you rely on, so you can decide when to update them, independently of any other package you may want to add in the meantime.

You could specify strict equality to package versions in your descriptor (aka depends: openwrap = 2.0.0), and for a project that only exists to build, say, an asp.net web app, that may be valid. If however you use your project to build a package, and the goal of OpenWrap is for one day everything to be a package, then you’re sharing your development-time dependencies with the world, and you prevent people from updating dependent libraries when they know they want to.

With locking, you can specify that a package, a list of packages or all of your packages are frozen in time. Trying to add, remove or update a locked package will simply fail.

Locking

Say I have a new project that depends on OpenWrap 2.0.

C:\demos\testlock> o list-wrap
 - Mono.Cecil (available: 0.9.4.1)
 - openfilesystem (available: 1.0.0.61263243)
 - openwrap (available: 2.0.0.81714117)

If a new version of OpenWrap becomes available, updating OpenWrap is going to be automatic.

C:\demos\testlock> o update-wrap openwrap -proj
Updating project packages...
Project repository: openwrap updated [2.0.0.84854337 -> 2.0.0.84856266].
...
Project repository: Package openwrap-2.0.0.84856266 anchored.

I can lock all my current packages by issuing the lock-wrap command.

C:\demos\testlock> o lock-wrap
Package 'openwrap' locked at version '2.0.0.84856266'.
Package 'SharpZipLib' locked at version '0.86.0'.
Package 'openfilesystem' locked at version '1.0.0.61263243'.
Package 'tdnet-framework' locked at version '2.0.0.48555719'.
Package 'Mono.Cecil' locked at version '0.9.4.1'.

Now all my packages are locked. Attempting to update any of those packages will do nothing. All the same, if I attempt to remove a locked package.

C:\demos\testlock> o remove-wrap openwrap
Cannont remove package 'openwrap' as it is currently locked. Unlock the package with 'unlock-wrap openwrap' first, then
remove.

Of course, you can decide to lock only one package with the –Name input on the command, and you can prevent dependent packages from being locked with –IgnoreDependencies. Check the get-help section on lock-wrap for all the details.

Unlocking

As simple as locking, the other way around, but I’ll just unlock openwrap and leave its dependencies locked.

C:\demos\testlock> o unlock-wrap openwrap -ignoredependencies
Package 'openwrap' unlocked.

Sharing locked packages

Once  you’ve added and removed locks on your solution, it is up to you to decide if you want those locks to be shared with your team or not.

The way to do this is very simple. A new packages.lock file has been added to the /wraps directory. If you want the locks shared, add it to source control. If you don’t want other team members to unlock, make the file read-only. Simples.

The feature you don’t want to use, ever

Finally, there is a hidden feature that allows you to declare a dependency in your descriptor to be strict (aka include the revision number). It was there all along but not usable unless you were the package manager itself, and now we’ve surfaced it, for those scenarios where you’re doing something stupid, you know about it and you want us to back-off with all of our convention, good of the ecosystem mantra.

You can declare a dependency using the Identical To unicode character, U+2261, more commonly known as . In your descriptor, you can then have:

depends: openwrap ≡ 2.0.0.84856266

And we’ll respect that. It’s not surfaced on the command line, and the use of the unicode character is to make it extra hard for you to do the wrong thing, while not preventing you to shoot your foot if that’s really your need.

Conclusion

As usual, if you have any question about how to use those features, please post them on http://stackoverflow.com and tag them with OpenWrap. If you have bug reports, please add them to http://github.com/openrasta/openwrap/issues. We are entirely community-focused and do the right thing for the software ecosystem. We can’t do that without you, users, so get in touch when you need to and we’ll try our hardest to help.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Rafki

    Man, have you even read semver spec? Or you are that kind of obsessive-compulsive “genius” that does not need no spec and “figure that out” just from the title?

    FYI semver defines three and only three segments of the module version: major, minor versions and patch. Plus optional label.

    Some times I’s seems you so busy cramming out code you forget to think.

    • http://codebetter.com/sebastienlambla/ Sebastien Lambla

      If you read the post, you’ll see “we ignore the revision” and “we’re friendly to semver.org”. In other words, whenever you set a dependency on the fourt part of the version, we just ignore it. If multiple packages have different revision numbers, we just assume that you built it multiple times and always take the latest. The point of this post was to provide an opt-out of that behavior when you depend on packages built for nuget .

      What exactly is it that you’ve seen in my post that would say otherwise, or imply that there is a fourth number in semver, or that i’ve not read the semver specification, or that I am an obsessive-compulsive asshole? If you let me know what your point actually is, I’ll happily update the post.

      As for semver’s labels, wee don’t support it on version identifiers mostly because our metadata will address that independently for providing the equivalent feature, although I don’t exclude just going for adding the label the way semver defines it, it’s not built yet, we’ll see. If you have an opinion (and you certainly seem like you have one), please continue voicing it and i’ll take it into account, like I do everything else.

      Thanks,

      the obsessive-compulsive genius that does not need no spec but actually read them.

      • Rafki

        Sorry for being rude in previous comment, but I still thing you do not grok semver and misinform everyone associating OpenWrap with it. Your understanding of versioning, I believe, comes from standard .NET/VS scheme of four sections. However you add fifth section for some reason:

        >> When you take dependencies on packages, we ignore the revision
        >> number (that’s the 5 [sic!] in1.2.3.4.5)

        The goal of semver spec is to standardize versioning scheme. Semver compliance requires package author to use version consisting *only* of three parts:

        * Patch version Z (x.y.Z | x > 0) which MUST be incremented if only backwards compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior.

        * Minor version Y (x.Y.z | x > 0) which MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MAY be incremented if substantial new functionality or improvements are introduced within the private code. It MAY include patch level changes.

        * Major version X (X.y.z | X > 0) which MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY include minor and patch level changes.

        So in your case (regarding subject of the article) you should ignore patch version (third part of the version) and tolerate autoupgrade in case of minor version (second one) change.

        BTW semver special versions (alphanumeric labels) are not metadata. v1.0.0beta1 and v1.0.0beta2 are two different versions according to the spec. So this functionality should be supported (I know, it’s kind of ugly, but it absolutely should) if you want to use word semver in relation to OpenWrap.

        Of course you always free to define your own scheme of versioning, but do not call it semver if it does not comply with the spec.

        Sorry again for previous comment’s tone.

        • http://codebetter.com/sebastienlambla/ Sebastien Lambla

          Let me clarify a bit more.

          OpenWrap is a package manager. It lets you define dependencies that take version ranges, the same way most package managers let you do that out there. I have argued that it is semver-*friendly*, or semver-*compatible*, not that it enforces or is a pure semver system. Saying OpenWrap shouldn’t be associated with semver is like saying gems shouldn’t either, seeing as they don’t enforce it either (as far as I’m aware).

          We encourage and recommend to library authors to follow semver.org versioning scheme (minus labels, I’ll get to that). If all library authors follow semver, and if users take dependencies in semver-friendly ways (aka `depends: myLibrary = 2.1` or `depends: myLibrary >= 2.1`), then all is good in the world. You’ll notice by the way that the spec provides exactly that guidance (and it’s my turn to quote):

          ” you can safely specify the Ladder dependency as greater than or equal to 3.1.0 but less than 4.0.0″.

          See, that’s what users do, use their package managers the way it can work, friendly with semver, if they so wish.

          .net has 4 components in a version. That’s not our decision, that’s the state of affairs. OpenWrap only takes into account the first 3 for dependency resolution, and let consumers make the decision of how they want to take dependencies. You could argue that we ought to just *enforce* semver, but we cannot, for two reasons I’ve highlighted:
          1. we don’t control what library authors do, and if we did we would be incompatible with the packages that are out there, and those don’t follow semver because…
          2. the other package manager for which people produce packages not only does not use or enforce semver, but on top of that they use 4 components to a version when we use 3.

          The whole package locking feature enables you to lock on the fourth component because we have to deal with the real world of packages that exist on nuget, and is a necessary compatibility feature because people don’t do semver. We can’t enforce semver because packages out there don’t follow semver.

          The best we can do is to provide the option of being semver friendly (we are, 3 components instead of 4, and you can define your own autoupdate mechanism that follows semver if you so wish). In other words, if you decide to build your libraries using semver then openwrap will function as a semver system, and if you have to use things that don’t do semver that works too.

          As for labels, we’ve not implemented them yet and I have a tendency to believe that they don’t quite solve what they are set to sort out. The current problem I have is that the specification in semver says that 2.0.0beta < 2.0.0 < 2.0.1, but I'm sure you can see that a beta package will probably not be compatible with the 2.0.0 public API the same way 2.0.1 would be. The solution I was going towards respects the logic of labels while solving this incompatible behavior, by enforcing that packagename+packageversion stays unique, and leaving the "beta" moniker as a namespace in which the package lives. If you don't import a beta namespace when importing your dependency, they won't show up, but your dependency versioning strategy stays the same. This also lets you specify versioning ranges as normal, and just specify which namespaces get imported (so you can chose to import beta packages, rc packages, etc).

          I'm not sure that it'll pan out or that supporting labels isn't the way forward, I'm undecided and the feature is just not implemented.

          Note that the semver spec doesn't enforce a resolution algorithm on package managers, but a versioning schemes on library authors. By being semver friendly, what I absolutely mean is that I want to stay as close as possible to the spirit of the spec, even if it means that some of the syntax is different (and I don't think the value in semver is the syntax).

          I hope this clarifies why I say that we are semver-*friendly* so far. You may disagree on the finer points, but there you are. If you want to continue chatting on the label issue, don't hesitate to email the openwrap development mailing list, comments are hardly the right place to discuss technical merits of various solutions.

      • Rafki

        I’m just wanted to emphasise my point again. If you want OpenWrap to be “http://semver.org-friendly system” it should not allow “revision number”. On the top it should use semantic associated with different version changes (basically by auto upgrading to lasted patch version when it’s available and tolerating minor version upgrades). If you want to use Nuget packages simultaneously forcing “people in the pit of success” you should distinguish between “do as you wish” versioned packages and OpenWrap “http://semver.org-friendly” packages.

        • http://codebetter.com/sebastienlambla/ Sebastien Lambla

          As I highlighted below (I just sum it up for anyone reading those comments), rubygems can be semver friendly if you set it to be semver friendly. Enforcing semver for some packages and not others is close to impossible to do reliably and enforcing semver all the time, because of how pacakges that exist outside of my control and guidance, is not possible at all, be it that I’m happy or not about it.

    • http://codebetter.com/sebastienlambla/ Sebastien Lambla

      re-reading the post, I seem to imply that I ask people to use revision numbers for patching, I’m not. It’s not that clear cut: revision is litterally ignore, it gives a bit of power to lib developers to force updates when needed. Lib develpers ought to introduce changes in behavior (including fixing bugs that may change behavior) by increasing the build, the way semver highlights. The issues we’ve had with library developers has been a change in revision where it should be on the build. The revision we use as the nuclear option which is the equivalent of forced updates on a windows box.
      I’ll edit the post to make that clearer and update earlier posts that date back before semver was how we decided to implement stuff (pre 1.0).

    • http://codebetter.com/sebastienlambla/ Sebastien Lambla

      I thought it’d be interesting to add for anyone that cares about this the latest rc of the semver spec, that adds revision and that we will support fully, as the current implementation of OpenWrap (since 1.0) is a subset of the now available full specification. Rafki, I’d highly suggest you go read the spec.

  • Pingback: The Morning Brew - Chris Alcock » The Morning Brew #956