Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

Chocolatey Now has Package Moderation

Well just after three years of having https://chocolatey.org, we’ve finally implemented package moderation. It’s actually quite a huge step forward. This means that when packages are submitted, they will be reviewed and signed off by a moderator before they are allowed to show up and be used by the general public.

What This Means for You Package Consumers

  • Higher quality packages – we are working to ensure by the time a package is live, moderators have given feedback to maintainers and fixes have been added.
  • More appropriate packages – packages that are not really relevant to Chocolatey’s community feed will not be approved.
  • More trust – packages are now reviewed for safety and completeness by a small set of trusted moderators before they are live.
  • Reviewing existing packages – All pre-existing packages will be reviewed and duplicates will be phased out.
  • Not Reviewed Warning – Packages that are pre-existing that have not been reviewed will have a warning on chocolatey.org. Since this is considered temporary while we are working through moderation of older packages, we didn’t see a need to add a switch to existing choco.

Existing packages that have not been moderated yet will have a warning posted on the package page that looks like

This package was submitted prior to moderation and has not been approved. While it is likely safe for you, there is more risk involved.

Packages that have been moderated will have a nice message on the package page that looks like

This package was approved by moderator mwrock on 10/26/2014.

If the package is rejected, the maintainer will see a message, but no one else will see or be able to install the package.

You should also keep the following in mind:

  • We are not going to moderate prerelease versions of a package as they are not on the stable feed.
  • We are likely only moderating the current version of a package. If you feel older versions should be reviewed, please let us know through contact site admins on the package page.
  • Chocolatey is not going to give you any indication of whether a package is approved or not. We expect this to be temporary while we review all existing packages, so we didn’t see much benefit to the amount of work involved to bring it to the choco client in its current implementation.

What This Means for Package Maintainers

  • Guidelines – Please make sure you are following packages guidelines outlined at https://github.com/chocolatey/chocolatey/wiki/createpackages – this is how moderators will evaluate packages
  • Re-push same version – While a package is under review you can continually push up that same version with fixes
  • Email – Expect email communication for moderation – if your email is out of date or you never receive email from chocolatey, ensure it is not going to the spam folder. We will give up to two weeks before we reject a package  for non-responsive maintainers. It’s likely we will then review every version of that package as well.
  • Learning about new features – during moderation you may learn about new things you haven’t known before.
  • Pre-existing – We are going to be very generous for pre-existing packages. We will start communicating things that will need to be corrected the first time we accept a package, the second update will need to have those items corrected.
  • Push gives no indication of moderation – Choco vCurrent gives no indication that a package went under review. We are going to put out a point release with that message and a couple of small fixes.

Moderation Means a Long Term Future

We are making investments into the long term viability of Chocolatey. These improvements we are making are showing you that your support of the Chocolatey Kickstarter and the future of Chocolatey is a real thing. If you haven’t heard about the kickstarter yet, take a look at https://www.kickstarter.com/projects/ferventcoder/chocolatey-the-alternative-windows-store-like-yum.

About Rob Reynolds

Rob Reynolds has been programming in .NET since the early days of 1.0... [Continue reading... at http://about.me/ferventcoder] On this blog you'll find just about anything I find interesting that has to do with development including Puppet and Chocolatey. Because we are all polyglot programmers in our own way, you may see anything from NHibernate to some custom CSS. Nobody is perfect and occasionally I may post something you find incorrect. Please keep me straight! With your help and interaction the community will benefit.
This entry was posted in chocolatey and tagged . Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Pingback: A Better Way of Installing and Updating Windows Packages – Simple-Talk

  • http://www.Redsandro.com Redsandro

    It’s kinda odd now but in the long run this will benefit the quality of chocolatey. I applaud this feature!

    I think though that certain pushers should be able to get the status of trusted pusher. After building some trust and a 100% trusted package ratio or something.

  • https://about.me/ferventcoder ferventcoder

    One thing we will do for maintainers is provide feedback on packages with approval for a one month grace period. Unless the package is new, broken, insecure or inappropriate.

    • http://www.Redsandro.com Redsandro

      One thing that would allow transparency for the audience and providing insight on the status of this process is an overview like this:

      Packages approved: 520
      Packages rejected: 35
      Packages queued for moderation: 1900

  • Simon Cropp

    you really should have done this with a grace period. ie one month of provide feedback but still auto approval.

    • https://about.me/ferventcoder ferventcoder

      What does this mean?

      • Simon Cropp

        so we have been happily deploying regularly and fixing issue with chocolatey for 7 months.

        Now we have a release blocked with some fairly insignificant feedback.

        To be clear we are happy to fix them and moderation is a good thing. But a softer introduction to the process would have been better

        • https://about.me/ferventcoder ferventcoder

          That’s probably something I might have done. Do you want to contact me by email and we can talk about which one it is?

        • https://about.me/ferventcoder ferventcoder

          I thought we were being pretty generous with our feedback, but perhaps we need to start off even softer. Like you mentioned, a one month grace period.

  • David Ebbo

    Rob, I’m not convinced this is a good direction. NuGet, Visual Studio extension gallery, npm and Ruby gems have managed to be successful without taking such steps.

    One problem is that it’s going to be pretty darn expensive to moderate everything in a vibrant ecosystem. Also, I really don’t think that packages can be ‘reviewed for safety’. How can you tell the difference between an exe that has malware and one that doesn’t? Whatever process you follow would be easy to foil.

    Anyway, it might make sense to update the post (or make a follow-up post) to cover these topics :)

    • https://about.me/ferventcoder ferventcoder

      Can nuget, vsix, npm or ruby have a script that runs as an admin that can call format c?

    • https://about.me/ferventcoder ferventcoder

      >One problem is that it’s going to be pretty darn expensive to moderate everything in a vibrant ecosystem.

      We have six moderators in different timezones and we plan to add more. Yes, there is a cost, but the damage of a malicious package has a much higher cost.

      Plus we’ve already increased the quality of around 10 packages just by having a chance to review and make suggestions to submitted packages. We have already rejected a package that is inappropriate for the community feed.

      >Also, I really don’t think that packages can be ‘reviewed for safety’. How can you tell the difference between an exe that has malware and one that doesn’t? Whatever process you follow would be easy to foil.

      I don’t disagree with this. But if we are ensuring that the package downloads from the official distribution point and not some other place, it’s not a special attack just through chocolatey (e.g. an equal opportunity attack for everyone).

      One thing we are moving forward on that is different from running out on your own and downloading from the official distro is that we have checksums and right not they are optional. But they won’t always be optional. So if someone switches out for an exe that is malicious at the official distribution point, the package will notice the checksums don’t line up.

      • https://about.me/ferventcoder ferventcoder

        An enhancement we will be doing for “safety” is to review the install in a sandbox in an automated fashion. Then we will know if the instructions work, and whether or not it installs additional malware. Also keep in mind chocolatey pro is going to do virus checking at runtime, so we are going to know the validity of the package.

        • David Ebbo

          I guess there are two kind of choco packages:

          1. Those that install popular tools like msysgit and Chrome. Here, the package owner doesn’t own the tool.

          2. Those that install custom tools written by the package owner.

          So I can see how the moderation will help with #1, as those things tend to be fairly simple scripts that just point to well known MSIs. So yes, I now agree that moderation is both achievable and worthwhile for those.

          But for the #2 kind, it’s a lot trickier. E.g. I have one such tool (https://chocolatey.org/packages/WAWSDeploy), and I don’t think it’s technically feasible to do a safety review on it. It’s an exe that the package itself brings in, as opposed to some well known public download. At that point, the only safety is knowing who wrote it, having other people pointing to it, etc… But does that mean it will always have the warning on it?

          PS: I swear that my package has no malware :)

          • https://about.me/ferventcoder ferventcoder

            For #2, I agree that it gets a little trickier, but I think we can get some of the benefit of autochecking for viruses out of it and if there is another location for this exe, we can verify the checksums match. If there is a public checksum for that version, we can also do a verification of the checksums. There are a lot more #1 packages than #2 at the moment. :)

          • David Ebbo

            [Weird my comment got removed. Re-adding it]

            Ok, if #1 is the main use case, then I’m with you that it’s a good direction. In my case, I chose to make it a Chocolatey-only release, so there is nowhere else to go find a checksum.

            There are other interesting things we could do for #2. e.g. allow me to vouch for my choco package on the Github repo (e.g. with some special syntax in the readme). This at least guarantees that the choco package was produced by the owner of the repo, which already goes a long way to build trust. Once on Github, it’s easier to tell if someone is real based on age of account, activity, …

          • https://about.me/ferventcoder ferventcoder

            I think we will see more #2, and with that we already have checksumming due to nuget package hashes being verified.

            So it just becomes a matter of the contents of the package being safe. It helps to know who the author is, that puts your reputation on the line. It’s easier to trust those you know. I do think there are definite options to vouch for the contents.

    • starrychloe

      Why not allow an independent testing lab to place their seal of approval on the package page, and let the package submitter pay the independent tester for testing their package and review?

  • Frans Bouma

    Any news how long this will take? I now have a banner across the package description that it’s not approved, which is kind of harsh, and makes it look like an illegal package, while it’s nothing of the sort. Also, if I push a new version in the future, do these new versions need re-moderation again? As that makes it a tedious process.

    • https://about.me/ferventcoder ferventcoder

      It’s going to take awhile to go back across 2300 pre-existing packages and moderate them. We just opened this feature on Saturday. We have already moderated around 500 or so packages including new submissions.