scriptcs – Living on the edge in C# without a project on the wings of Roslyn and Nuget

MP900430605[1] 

Disclaimer: I am not the first person to create a tool for C# scripting. There are a long list of projects from too many people to name, and I respect all of them :-) A big part of why I did this was also to mess with Roslyn.

Learnings from node

For the past 18 months I’ve been living mostly in text editors and script getting fully absorbed in node.js. Recently I started to think a lot about my C# development experience and if there are things I could bring back from my learning’s with node / JavaScript

  • No projects, just script- One of the things I love about node.js is you don’t need a project. You can just jump in a folder start creating js files and go to town.
  • No IDE requirement, you can just use a text editor.
  • Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.
  • No compilation – This is a big one. With node, I just run node.exe and my app and it works. I don’t have to first create an executable to run, I just run.

On first glance it might seem like this is just oil and water with C#.

Thanks to Roslyn and Nuget however I no longer think that’s the case.

What is Roslyn?

image 

Roslyn is a compiler as a service. It gives you an API that lets you compile and analyze code on the fly. Additionally it gives you hooks to dynamically refactor and generate code. Beyond that it also includes really powerful scripting support allowing you to author applications and components as loose C# scripts. You don’t need a class if you don’t want one and you don’t need to compile. And you don’t need to use Visual Studio, you can host Roslyn in your own applications and components and it’s even got nuget packages.. You can read more on Roslyn at MSDN or in a nice post from Filip on Roslyn and Web API

Roslyn is a breath of fresh air in terms of it’s simplicity. If you have worked with the existing horrendous mechanisms for doing this like the Codedom you are going to love Roslyn. You can literally be up and running with Roslyn having it compile scripts for you in a matter of minutes.

How does Roslyn help?

I can cross off three bullets :-)

  • No projects, just script- One of the things I love about node.js is you don’t need a project. You can just jump in a folder start creating js files and go to town.
  • No IDE requirement, you can just use a text editor.
  • Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.
  • No compilation – This is a big one. With node, I just run node.exe and my app and it works. I don’t have to first create an executable to run, I just run.

Now yes it does compile, but that’s all taken care of for me, I don’t have to think about it.

Packages vs Assemblies

For the last bullet I looked to nuget. The nuget command line provides an easy way to install packages right from the command line into the same folder as a set of scripts. However, the challenge is how to make those scripts see those packages? Today Nuget works great with a csproj and within Visual Studio or using MS Build. But what if you don’t have Visual Studio or a csproj? Well when you install packages using the cmd line you get folders for each package that at some level actually do contain binaries. So, if you can find those binaries and get them loaded up into Rosyln, you are golden. Yes this doesn’t get rid of assemblies, but I don’t have to manage them individually, I just install packages.

  • Packages over assemblies – In node, when you want to get something you use npm to download the packages.It’s super simple. You just have your app and your local node_modules folder and you are good to go.

Roslyn + Nuget = scriptcs

Having crossed off all my bullets, I then decided to spike a little with Roslyn and out came Scriptcs: https://github.com/glennblock/scriptcs

scriptcs is a little command line tool which lets you have the following work flow.

  • You author your “app” as a .csx file or csharp script in your favorite editor. There’s no project.
  • You add dependencies to your app by simply installing nuget packages from the command line. By convention the dlls within those packages will be loaded up. No need for references as the idea is if it’s there you want it.
  • You add GAC references or other dll references (not nuget packages) by using the “r:” Roslyn syntax, i.e.”r: System.Net”.
  • You run your app using “scriptcs” passing the csx file.

For example here is a snippet of setting up a web api host.

Back to reality, though the future looks bright.

MP900403166[1]

At this point I would say this is just an experiment. I am not claiming you can build an enterprise app with this or anything like that. I am using this more as an exploration to understand what a c# script experience from outside VS could look like.

From here there’s a lot more places to go. One would be allowing you to have includes of other csx files as you probably don’t want your whole app in a single file! Beyond that Roslyn offers a ton of rewriting, AST analysis or even auto-refactoring capabilities that I am sure can be leveraged in scriptcs but are not.

Seeing this definitely makes me excited about where things can go in the future. I’d love to hear your thoughts.

Posted in coding, rosyln | 11 Comments

Teach your Azure Website new deployment tricks with azure-cli 1 of XXX: Running Mocha Tests

Disclaimer: This post is based on content I previously posted here and which should really be its own series.

Every time you deploy your Website, the server runs a bunch of steps which vary depending on the application. If it is a node.js application for example it will run npm if it sees a package.json present to install node modules.

Wouldn’t it be cool (and useful) if you could customize that step to do whatever you want, like run mocha unit tests? If you said yes, then you don’t have to wish any longer, it’s here.

To customize deployment you use a new command on the azure-cli, “azure site deploymentscript”. This will generate for you a script either in .bash or .cmd format which contains all the server logic that will execute when your site is deployed. You can then easily customize that script to your heart’s content.

For more details on how this command works and further examples including with an MVC app, check out Amit Apple’s nice series.

When you use this feature, you really are too cool for school!.

To see it in action, I am going to show you how to enable a common scenario (and timely based on my last Mocha debugging post), that is make my mocha scripts run for my node app every time I push via git. Assume I created a simple express hello world by running the “express” cli.

I then create a dummy test.js which will always fail.

Now in my app folder, I run the deploymentscript command specifying a node app and to create a bash script.

image

You can see the generated script here in this gist: https://gist.github.com/4331260/revisions by looking at the last file.

If you look at the script you’ll see several different sections.

  • #Prerequisites – This section defines code that should run before anything else. For a node app here is where it detects if node is installed.*
  • #Setup – This handles setting up script variables or installing other necessary modules. For example you’ll see it installs the node “kudusync” module which is used for moving files from where the files are pushed to the target website.
  • #Deployment – This handles actually deploying the code (calling kudusync) and other steps like calling npm for node modules.

*Note: There is a bug currently in the bash version of this, which uses the “where” command to find node which does not work on bash. The fix is in the gist above at line 24.

One great thing about this script, is it is designed to actually allow you to run it locally. It detects whether you are in the local folder and will just create an artifacts folder for the files that are copied over. This is really useful when developing.

For my unit test scenario I want to do things a little differently than the generated script does by default.

  • I want it to install mocha, the same way it installs kudusync
  • I want it to run npm before it copies the files to the website, not after.
  • I then want it to run mocha and if the tests fail I want the deployment to abort.

Doing that means moving a few things around and adding some new steps. In order to make it also work locally there’s some light mental gymnastics, but nothing rocket science.

The final script is the top file in the gist revisions (https://gist.github.com/4331260/revisions)

I’ve annotated the parts I changed with “# gb:” and thanks to github’s awesome diff feature you can easily see the comments and what was changed.

Now when I deploy the site, it runs my tests and I get the output right when I git commit.

image

Notice above that my tests failed and my website was not updated.

This is just one of many scenarios that are now opened up to you through the new custom deployment feature.

In my next post in the series, I’m going to attempt to show you how to use this to download any custom node version you like the first time you deploy. I don’t know if this is really going to work, but it does in the blackboard of my mine, we’ll just have to wait and see what happens!

Posted in azure, node.js | 1 Comment

Debugging mocha unit tests with WebStorm step by step

WebStorm is a pretty awesome IDE (there Hadi, I said it!) which I do not use enough! As a node.js developer one of the great features it offers is a really nice debugging story, something that MANY node developers die without long for. It also has support for various test runners like nodeunit and mocha. Can we put those features together though? I mean can we debug our mocha unit tests in WebStorm? I’ve been wondering this for a while and just the other day Josh and I were chatting about it.

After a bit of investigating, I can say the answer is YES. If you search around the internets you’ll find a post which describes how to do it. In this post I’ll show you step by step exactly how to do it with screenshots :-)

Prereq – Install mocha

npm install mocha –g. (use sudo if you need to)

Step 1 – Open your project

Launch WebStorm and then open the node project that includes your mocha tests.

Screen Shot 2013-01-16 at 10.32.45 PM

Step 2 – Create a new Run/Debug configuration

Click on Run->Edit Configurations in the menu or select “Edit Configurations” from the configuration drop down. Then press the + to add a new configuration and select “Node.js”.

Screen Shot 2013-01-16 at 10.39.54 PM

Step 3 – Enter the configuration information

Now there’s a few pieces of information you need to enter.

  • Path to Node: Should be set by default, but if you have a specific node version you want, see it here.
  • Path to Node App JS File: This should point to where your mocha module is installed globally. It’s very important that you point to the _mocha file as you can see below.
    • On Mac / Linux it will be in /usr/local/lib/node_modules/mocha.
    • On Windows you’ll find it in your %APPDATA%/npm/node_modules/mocha folder.
  • Application Parameters: If you want to debug a specific file (rather than running all) put the name of the file you want to debug (cli-buddy-tests.js in the example) as well as any additional mocha params.
  • If your tests rely on any environment variables, set those in the “Environment Variables Window”

Screen Shot 2013-01-16 at 10.01.59 PM

Step 4 – Add a break point and go!

Add a break point for where you want to start debugging. Then hit the debug button are you are on your way.

Screen Shot 2013-01-16 at 10.01.15 PM

As you can see above I hit my breakpoint. I can now step, see variables, etc.

All I can say is booyah.js!

Posted in node.js, webstorm | 4 Comments

New OSS VM Image Depot for Windows Azure – A big step for VM kind

MS Open Tech recently launched their new portal for OSS VMs. What an awesome discovery on my 7th anniversary at Microsoft! 

With the new portal 3rd parties you trust can contribute images for all sorts of server products like RIAK and Ruby stack!

When you go to the site (http://vmdepot.msopentech.com) you’ll see this screen which lists the images:

clip_image001

Click on one of the images, say RIAK and you get this screen.

clip_image002

Now I want to install RIAK which as you can see has been published by Basho. I just click on that deployment script link and get the commands I need to use our CLI to deploy it! MS Open Tech added the “-o” command to our “azure vm create” to enable this to work.

This is a huge step, and opens up a ton of new opportunity for customers and for 3rd parties to benefit from Azure!

Great work MS Open Tech!

I love this company!

Posted in azure | Leave a comment

Simple Powershell scripting with Azure Powershell cmdlets

In my last post I covered how you can write bash scripts using the azure cli. Right after posting, @dfinke pinged asking rightfully “What about Powershell? Can I do the same things?”.  Yes Doug, absolutely! In this post I’ll show you can do the same and more in Powershell using our Azure Powershell cmdlets.

Scripting in Powershell

To understand scripting in Powershell, there’s a few things you need to know.

  • Powershell cmdlets write (or should write) objects to the pipeline, not just output text. This is really one of the major differences, you are not writing scripts that have to parse text, deal with regexes etc just to do basic things. Instead you just wire up properties. If you compare to my previous post, you’ll see instantly how this manifests itself and makes life easier.
  • Powershell cmdlets are very fine-grained each performing a specific purpose. You can chain together different cmdlets by piping objects from one cmdlet to the next. You can learn more about piping and the pipeline (and should if you don’t know about it) in Powershell here.
  • Pipelining is a whole topic on it’s own, but basically as long as objects coming from the pipeline have properties that match property names on the destination cmdlet, it should just work.
  • You can create local variables and save objects from previous calls which you can reuse. This is really nice for scripting.

To script with our cmdlets, you will basically call one of the cmdlets and either save the result into a variable that you can pass in, or pipe it directly. In my examples below, you’ll see that in most cases, you just pipe directly.

Below is a list of some common simple scripting scenarios for websites, which includes the cmdlets and a small explanation of what each is doing. This should get you started and you can use a similar approach to work with all of our cmdlets.

Disclaimer, I am not at all a Powershell expert, so there probably are better ways to do what I am showing you.

Stop all running sites

  • Retrieves all websites, this returns a collection of Site objects. Each object is a summary object which has a Name property which is the site name.
  • Pipes to Stop-AzureWebsite (matching Site.Name to the Stop-AzureWebsite Name parameter) which stops each site.

Start all stopped sites

  • Retrieves all websites
  • Pipes to Start-AzureWebsite which starts each site.

Delete all sites

  • Retrieves all websites
  • Pipes to Remove-AzureWebsite which removes each site and passes the force param to force removal without prompting.

Set an app setting to every site

This one may seem a bit strange at first, but read the explanation. You can really see Powershell’s object oriented nature shining through here :-)

  • Retrieves all websites
  • Pipes to For-EachObject which loops through all the objects.
  • The –Process param specifies a script to execute for each object. Within the script $_ refers to the current object.
  • The script first calls Get-AzureWebsite to get the full Website object passing the piped in name which is stored in a $site variable. When Get-AzureWebsite is called for a specific Website it retrieves all the details including the settings.
  • Next it updates the settings for $site with a “Bar” entry. This is important because we don’t want to lose existing settings we just want to update/append.
  • Finally it calls Set-AzureWebsite passing the current site name and the new settings object which contains the old settings and the new Bar setting.

Getting cmdlets to run in the background.

All the previous calls are blocking. Powershell does support the Start-Job cmdlet which allows you spawn things off in the background, so you can modify any of the script to run in the background. This took me a bit of playing, but I finally got the last one working (which is the hardest).

Hold on to your saddles folks! If you thought the last one was a little tough, feast your eyes on this!

  • Retrieves all websites
  • Pipes to For-EachObject which loops through all the objects.
  • The script calls Start-Job to launch a background job
  • The –InputObject specifies to pass the site name as a variable to the job script.
  • The –ScriptBlock specifies the script which executes within the background job (Matrix anyone?)
  • The job script first calls Get-AzureWebsite to retrieve the site info and stores it in the $site variable.
  • Next it sets the the settings.
  • Finally it calls Set-AzureWebsite passing the current site name and the new settings object which contains the old settings and the new Bar setting.

This one actually deserves a screenshot or two, so here goes. Below you can see running the script which outputs the list of created jobs.

image

Now you can see that it has created for me a job for each website that was retrieved. The jobs are running in the background. I can use the Get-Job command to monitor the status of any jobs running in my shell.

image

As you can see above all the jobs completed a few seconds later. There is also a Receive-Job command that you can pass a job to in order to get more details and I believe the Verbose output. In this case Set-AzureWebsite doesn’t return anything, however I can check one of my sites and see that it worked. Notice Bar1…

image

Merry posh scripting

As in my previous post, this is just the beginning of what you can do once you start mastering scripting in Powershell. Just go ask my friend and Powershell guru Peter Provost. We are doing a lot of deliberate up-front thinking about scripting scenarios with our Powershell cmdlets as there are existing mechanisms since it is so core to being a first class Powershell cmdlet.

Let us know if there are areas you think we need to improve the scripting support, and please share links to any cool Azure powershell scripts you have authored!

Happy Holidays!

Posted in azure, powershell | 6 Comments