Discovering Web.configs for Deployments with MsBuild Custom Targets

My current job is working on a web application that needs a lot of configuration, and is deployed to multiple environments. We’re using web.config transforms to apply the configurations, but our build pipeline (Team City and MsBuild scripts) builds a deployment “package” as a release candidate, then a subsequent build step deploys (or redeploys) that package to test, staging and production environments (or doesn’t, if we don’t like the release candidate at any point). When we deploy, we ship the site plus one of around 20 transformed web.configs.

So while getting automated deployments up and running with MsDeploy, one of the things I needed to do was create a portable “white label” deployment package, then set the configuration to whatever environment I deployed to. As I’m using web.configs, I need to do the following:

  1. Generate all possible web.configs when I’m building the package to get a set of build artefacts consisting of zipped web application package plus transformed web.configs.
  2. Publish the site, then apply the appropriate web.config (from build artefacts) at deployment time.

So what I’m about to show is half “quick and dirty solution to my deployment problem”, and half “crazy things you can do with MsBuild”.

Publishing The Chosen Config

Using an MsBuild file to wrap my build and publish commands, I have this big chunk of MsDeploy for deploying my web.config to my Azure or IIS site:

<Target Name="_DeployWebConfig">
 <Exec Command='$(MsDeployExe) -source:contentPath="$(WorkingDir)$(PackageDeploymentDir)\Web. 
$(Configuration).config" -dest:contentPath="$(AzureSite)\web.config",ComputerName="https:// 
$(AzureSite)",UserName="$(AzureUsername)",Password="$(AzurePassword)",IncludeAcls="False",AuthType="Basic" - 
verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension - 
retryAttempts=2 -verbose -userAgent="VS12.0:PublishDialog:WTE12.3.51016.0"'/>

That’s all the MsDeploy for this post. We know we can ship the configs. First we need to get them.

Create All The Configs

My first attempt at transforming all the configs (from web project directory into the deployable package directory) is to include the “TransformXml” task at the top of the build script:

<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v10.0\Web 

And then have a target I can call to transform the configs.

<Target Name="_TransformWebConfigs1">
 <TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Debug.config" 
Destination="$(PackageAssemblyDir)\Web.Debug.config" StackTrace="true" />
 <TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Test.config" 
Destination="$(PackageAssemblyDir)\Web.Test.config" StackTrace="true" />
 <TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Release.config" 
Destination="$(PackageAssemblyDir)\Web.Release.config" StackTrace="true" />

Which is OK if you only have a few configurations. But what if you had to manage several environments? Let’s expand this target:

<Target Name="_TransformWebConfigs2">
 <Config Include="Debug"/>
 <Config Include="Test"/>
 <Config Include="Release"/>
 <TransformXml Source="$(WebApp)\Web.config"
 StackTrace="true" />

Not much better, except now what we’re doing is creating an item group that lists all our configs, then using MsBuild’s “%” operator to iterate though all the items in our item group. Yes, that “TransformXml” call is basically a “foreach” loop.

Find All The Configs

What we really want is to find all the configs automatically. This is where we start to hit the limits of MsBuild (or at least my knowledge, I might have missed something in the reference), because we want to get a flattened directory listing of the configs.

Consider discovering the configs and loading them into an ItemGroup:

<Target Name="_DiscoverConfigs1">
 <!-- Finds all config files, BUT keeps webapp path pre-pended -->
 <Resources Include="$(WebApp)\Web.*.config"/>
 <Message Text="[@(Resources)]"/>
 <Message Text="All configs:" />
 <Message Text="[%(Resources.Identity)]"/>

What we’d like is an ItemGroup with just “web.Debug.config”, “web.Test.config”, etc. I can’t find an easy way to do this using MsBuild syntax. But hey, MsBuild can run anything. So we could run this through a little script or even a tiny console app. Or just write C# directly in our build file. Yeah, let’s do that

First, add another property. We’re going to be creating our own MsBuild task.

<MSBuildPath Condition=" '$(MSBuildPath)'=='' ">C:\Windows\Microsoft.NET\Framework 

Now, let’s rewrite our discovery target:

<Target Name="_DiscoverConfigs2">
 <!-- Finds all config files, BUT keeps webappPath pre-pended -->
 <Resources Include="$(WebApp)\Web.*.config"/>
 <Message Text="[%(Resources.Identity)]"/>
 <_FlattenPathsTakeFilenamesOnly Filepaths="@(Resources)">
 <Output TaskParameter="Filenames" ItemName="WebConfigs"/>
 <Message Text="[@(WebConfigs)]"/>
 <Message Text="All configs:" />
 <Message Text="[%(WebConfigs.Identity)]"/>

And that “_FlattenPathsTakeFilenamesOnly” task? Let’s create it. Time to write some basic C# directly in our build script.

<!-- Create custom task to get file containing a version string in a directory -->
 <UsingTask TaskName="_FlattenPathsTakeFilenamesOnly" TaskFactory="CodeTaskFactory" 
 <Filepaths ParameterType="System.String[]" Required="true"/>
 <Filenames ParameterType="Microsoft.Build.Framework.ITaskItem[]" Output="true"/>
 <Using Namespace="System.IO"/>
 <Using Namespace="System.Linq"/>
 <Code Type="Fragment" Language="cs">
 Filenames=new TaskItem[Filepaths.Length];
 for(int i=0;i < Filepaths.Length;i++)
 Filenames[i]=new TaskItem(Filepaths[i].Substring(Filepaths[i].LastIndexOf("\\") + 1));
 //Console.WriteLine("Split " + Filepaths[i] + " > " + Filenames[i]);

This is a trivial example, if you were building a custom task that really did something, write and compile it elsewhere (eg. as part of your solution).

So our final config discovery target looks like:

<Target Name="_TransformWebConfigs3">
 <Resources Include="$(WebApp)\Web.*.config"/>
 <Message Text="[%(Resources.Identity)]"/>
 <_FlattenPathsTakeFilenamesOnly Filepaths="@(Resources)">
 <Output TaskParameter="Filenames" ItemName="Configs"/>
 <TransformXml Source="$(WebApp)\Web.config"
 StackTrace="true" />

This just trawls through your web app source directory, finds all the source web.configs, runs each in turn through the transformer and dumps into your chosen directory. Which for our requirements, gives us all the possible web.configs ready in case we want to deploy one of them.


Pre-Transforming Web.Configs For Deployment

When your approach to automated deployment is to build a deployment package, then later deploy that to successive test/staging/live environments, you need to manage configurations. Unfortunately, Microsoft’s web.config approach is designed to work only as part of a configure-build-deploy process. For configuring at deployment time, you need a better approach (parameters files). Or a hack… There are better ways to manage configurations, but you might not have the ability to move away from web.configs.

A quick note: I originally included this section as part of a larger post on MsDeploy, but really this is a separate subject.

Transforming Web.Configs at Build Time

I’m going to manually add a target to my MsBuild file to transform my web.configs. I don’t know at build time where my deployment will end up, so I just include all of the possible configs.

We need a solution with an MsBuild file, and the necessary build configurations and web.config transform files setup. When building the deployment package, we also transform all the possible configs (and include in the deployment artifact). When deploying, we MsDeploy the entire site (from zip package), then do an additional MsDeploy of a single web.config file.

Now add a “_TransformWebConfigs” target and a reference to the “TransformXml” task that I’m going to borrow.

Right at the top of your build file, inside the Project element and before the PropertyGroup, add this:

<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.Tasks.dll"/>

For now I’ll just go process each configuration to create all possible web.configs in our package, transformed and ready to deploy. I’ve named the files “web.configurationName.config”, you could store them in “ConfigurationName\web.config” instead.

<Target Name="_TransformWebConfigs">
<TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Debug.config" Destination="$(PackageAssemblyDir)\Web.Debug.config" StackTrace="true" />
<TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Test.config" Destination="$(PackageAssemblyDir)\Web.Test.config" StackTrace="true" />
<TransformXml Source="$(WebApp)\Web.config" Transform="$(WebApp)\Web.Release.config" Destination="$(PackageAssemblyDir)\Web.Release.config" StackTrace="true" />

And to call this, extend your “BuildPackage”, between building the package and copying to the deployment directory, add this:

<CallTarget Targets="_TransformWebConfigs" />

Now we add a “_DeployWebConfig” target that applies the correct config based on the $(Configuration) property (call MsBuild script with “/t:DeployPackage /p:Configuration=Test”). This is almost like the previous MsDeploy command, except:

    1. For the source, it really doesn’t like relative paths, so I grab the current working directory
    2. I’m explicitly setting the destination, because we’re renaming the file as we deploy
    3. You don’t set the “IIS Web Application Name”, MsDeploy complains if you do
<Target Name="_DeployWebConfig">
<Exec Command='$(MsDeployExe) -source:contentPath="$(WorkingDir)$(PackageDeploymentDir)\Web.$(Configuration).config" -dest:contentPath="$(AzureSite)\web.config",ComputerName="https://$(AzureUrl)/msdeploy.axd?site=$(AzureSite)",UserName="$(AzureUsername)",Password="$(AzurePassword)",IncludeAcls="False",AuthType="Basic" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -retryAttempts=2 -verbose -userAgent="VS12.0:PublishDialog:WTE12.3.51016.0"'/>

And you just insert the call to deploy the config at the end of your “DeployPackage” target:

<CallTarget Targets="_DeployWebConfig"/>

Note that your “web.X.config” gets renamed to just “web.config” on the target server during the deployment.

Automating Web Application Deployments to Azure with MsDeploy

We all know that having developers manually deploy code to test or production from their laptops is a really bad idea (wastes time, leads to unrepeatable “mystery” builds, hard to ensure you ship the approved and tested code, no traceability when things go wrong, etc.). So unless you’re willing to endure the three-monthly waterfall pain of spending days building and deploying the one true release version of your code, one of the first things you should do with a new project is set up a continuous integration (CI) and automated deployment system.

Microsoft make some great tools, but sometimes they make them really hard to use. A great example is their application deployment tool “MsDeploy”. Especially when it comes to Azure websites. When you’re manually deploying from Visual Studio then it’s easy, you just tweak the publish profile in a wizard and then hit “publish”, but as soon as you try to script and automate things, you run into all sorts of fun trying to figure out the MsDeploy syntax and error messages.

When my current project kicked off, we made the decision to host with Azure, and then found that setting up automated deployments to Azure is complicated. Because “MsDeploy”. So this is a guide, based on my experiences, on how you might go about automating your deployment process. It’s written with the aim of deploying a .Net MVC web application to Azure, but you can probably adapt for other purposes. I’ll also try and highlight any differences if you’re deploying to IIS instead.

Warning: this is the “works on my machine” version of this guide. I can’t guarantee that you won’t run into problems with your site, but I have successfully repeated this experiment on a couple of applications (including the one I actually get paid to build). So my aim here is not just to give you a bunch of MsDeploy command syntax, but to show how you can start to build and troubleshoot your own deployment process.

The Goal – Automated Build and Deployment

So this was all driven by the needs of our build and deployment process, which we want to be fully automated, and have an initial “continuous integration” build generate a versioned “release candidate” package that we can then use to deploy to successive test, staging and production environments (with the appropriate configuration). We’re agile, and we want to be able to deploy frequently to test and production, so we need an automated process.

In the real world, our product ships as multiple applications, to multiple production environments (different configurations). But we’ll start with the assumption that you just have a single web application to be deployed to “dev”, “test” and “production” environments.

So my inital aim is to eliminate Visual Studio from the process and get everything running on the command line. Those commands can then be scripted or otherwise run by the build server.

The Tools

First you’ll need Visual Studio (I built these examples with Community 2013). Thoughout this tutorial, I’m also going to be using MsBuild and MsDeploy on the command line. Oh, and you’ll need an Azure account (they do free trials). I’ll assume familiarity with VS and Azure, but MsBuild and especially MsDeploy might be new.

I’m not going to cover build servers, but instead just cover the process of escaping from Visual Studio to the command line. You should be able to get any build server to run these commands. And while I’m using MsBuild scripts to simplify some of the commands, you could do this another way. For the actual MsBuild and MsDeploy commands, I’m just going to call the .exes directly from my script, as you would on the command line. (Note that MsBuild has a built-in MsBuild task you can use, there can be complications getting this to use a suitable version of MsBuild for packaging and deploying).

By scripting deployments with MsBuild, I’m achieving a few things:

  1. Hiding command complexity (you tell the script to deploy and it handles all the MsDeploy arguments)
  2. Keeping complexity with the codebase – the build server calls a simple logical operation (“deploy to test”) and the build script (stored in source control with the code it builds) handles the details, making adding applications/artefacts to the build fairly easy
  3. Giving me an easy way to create and debug deployment operations locally without having to edit live on the build server

Finally, any version numbering is between you and your build server, I’m just assuming that we want one so we can track different releases (and track the current test/staging/production back to the code in source control) by setting the version number in AssemblyInfo files (at build time) then displaying somewhere (footer or about/diagnostics pages) as a quick check that the build and deployment actually worked.

The Approach

My basic approach to this has been to:

  1. Take the reference (manual) Visual Studio action, then script it
  2. Wrap commands in an easy-to-use format (MsBuild targets)
  3. Get a command working for hard-coded values before making it configurable
  4. Study errors carefully, check for stupid syntax errors, go slowly, be prepared to re-visit previous steps

This post steps through the process of getting a new web application to the point where it can be assembled into a deployable package, then deploying that package, without requiring Visual Studio. There’s some chunks of sample code (expect about 80 lines of MsBuild) for you to copy-and-paste and customise.

The Manual Publishing Process

If you manually publish to Azure or IIS from Visual Studio, you’re already using MsBuild and MsDeploy. Your process is probably something like:

  1. Create new build configurations, web.config transforms and publish profiles as necessary
  2. Make changes, test locally (uses MsBuild to compile code)
  3. “Publish” your project straight to Azure (builds, assembles what needs to be deployed, then uses MsDeploy to ship code to Azure)

So we want to take this process and automate it with some kind of build script, calling MsDeploy from our fancy new script.

Note: you can also publish your site to a package within Visual Studio and then deploy that later. If you’re already doing that, you’ve probably solved the hard part of deployments (actually deploying things).

A quick introduction to MsBuild and MsDeploy

MsBuild is Microsoft’s build engine. It’s what runs when you compile code in Visual Studio. But you can also run it on the command line, and feed it your own build scripts. In fact, you’re already doing that – a “.csproj” file contains MsBuild commands (XML).

MsDeploy is Microsoft’s deployment engine, and it’s what you’ll use to get your app deployed to Azure. In this scenario, it’s basically doing a “smart file copy” that allows the client (that would be us) to ship files (or at least changes) to the server (Azure), and set a few permissions along the way. MsDeploy can be run two ways – from Visual Studio, or from the command line (note that VS runs its own copy of the MsDeploy engine).

Warning: MsDeploy is painful to work with. Documentation is poor, the error messages are unhelpful, and taking a working set of commands for one solution and transplanting them to another may throw up all sorts of problems.

When you publish a site in Visual Studio, you’re running MsDeploy. You create a “publishing profile” (look in your web app project’s “Properties/PublishProfiles” folder), and then you publish with your chosen profile and configuration. There are basically two ways to publish an application in Visual Studio:

  1. The “Build and deploy” approach – compile the code, configure (web config transforms) and deploy as a single operation.
  2. The “Package” approach – compile the code and zip into a package, to be deployed at some later date.

We’re interested in creating and deploying packages, but we’ll be doing the “build and deploy” as an exercise.

I’m going to try and go through the steps that get you from a manual “build and deploy” to a fully automated system of building a package, then repeatedly deploying it to one environment after another.


OK, so we need an application to deploy, an Azure site to deploy to, and an MsBuild script to hold the deployment commands. You can probably skip/skim these next sections.

Create a basic web app solution

You could go straight into automating your current project, but I’m going to use a really simple web app (called “DeployableWebApp”) for this example, you may want to do the same.

So open Visual Studio, and create a new Web Application. It doesn’t matter what you put in here, but I’d recommend you have the following:

  • A message or other output that requires compilation to change (to show it’s a new build)
  • A message on a view that you can change for each deployment (to show that the deployed code has changed)
  • A web.config setting representing the environment that you can display somewhere – we’ll use this to test applying the configuration at deployment time

Create an Azure webapp

Log into your Azure account and go create a new web application. Call it anything you want (mine is called “deployme-dev”). For now we only need one application for “dev”, but later we’ll try deployments to “test” and “production”. You’ll also want to go and download the publish profile file, which contains some settings we’ll steal later.

Create an MsBuild script

You can just put your MsBuild targets and properties into your “.csproj” file, but I prefer to keep my build scripts in separate files. Add a new solution item, an XML file called “WebDeploy.msbuild” (you can call it anything you want, including the file extension). And then create your most basic build script:

<?xml version="1.0" encoding="utf-8"?>
<!-- MsBuild script for using MS Deploy - must be run as admin! -->
<Project xmlns="">
<!-- Add properties to configure stuff -->
<!-- Add targets to do stuff -->

For this example, I’m using one MsBuild script for both the building/package and deployment operations. You might want to split these up, as they will ultimately run on different build/deployment processes.
As your MsBuild file grows, you’ll want to organise things. You can expect your build file to contain:

  • References to external tasks
  • Properties (for configuration)
  • Build targets
  • Deployment targets
  • Internal helper targets

Run a test command from an MsBuild target

Add this into the build file, in the “Project” element:

<Target Name="HelloWorld">
<Message Text="I am the thing that deploys other things" />

To run this, you need to open a command prompt, navigate to your solution directory, then type:

msbuild WebDeploy.msbuild /t:HelloWorld

This passes your script to MsBuild and calls the “HelloWorld” target. But… unless you have “msbuild.exe” on your path, nothing will happen. So go edit your environment variables.

Or, try this trick – create a custom command prompt text file named “CommandPrompt.bat” in your solution directory, and make it look like this:

CD /D "%~dp0"
@%comspec% /k "SET PATH=C:\Windows\Microsoft.NET\Framework\v4.0.30319;%PATH%"

Tweak the paths, add your own extra tools etc. Now you can fire up your command prompt and msbuild.exe is on the custom path (which is used for commands run in this command prompt only).

Your general MsBuild-running syntax is

msbuild path-to-build-file /t:TargetName /p:Properties

You can specify multiple properties with either “/p:prop1=X /p:prop2=Y” or “/p:prop1=X;prop2=Y”.

Important note: for running MsDeploy you’ll need to be running the command prompt as Administrator.

Manual publishing with “build and deploy”

First we test that a manual publish in Visual Studio works and we can actually deploy to our Azure site.

Select your site and go to “build->Publish”, and create a new profile called “azure-publish-dev”. On the connection tab, set method to “web deploy”, and raid the publish profile to set server to the publish Url (ends “:443”), site name to msdeploySite, username and password to the generated values. For now, you can ignore the “settings” tab, just go and publish. Hopefully this works. Go change something and re-publish to prove it.

So what this actually did was the following:

    1. Built your application (look in the “project/obj/config” folder)
    2. Figured out what needs to be included (look in “project/obj/config/Package” folder, especially the “manifest.xml” file)
    3. Ran MsDeploy to copy the changes

Look in the build output window for more details. Something you’ll notice is a line like this:

Start Web Deploy Publish the Application/package to " ..."

And yes, that command is partially-hidden. Welcome to MsDeploy. It can be handy to know what Visual Studio is doing when it publishes. Unfortunately, Visual Studio has its own way in to the MsDeploy engine.

MsDeploy and command discovery

So by default, Visual Studio is accessing the MsDeploy assemblies directly. But you can switch it to use the command-line executable instead. Basically, you just create a “MyWebAppProjectName.wpp.targets” file in your project (it’s an MsBuild file that gets run during the build), and use it to set the “UseMsdeployExe” property to “true”. Best add to your solution so VS finds it. Full details here:

Anyway, you should now see the full msdeploy.exe call in the build output. That’s going to form the basis of everything else we do. Start by copying what Visual Studio does. Then we can adapt to other deployment scenarios.

You can re-run this command as a test. Either paste straight into a command prompt, or paste into a new text file (call it “deploy.cmd” and run by typing “deploy”). You’ll need to make sure anything with spaces (eg. the msdeploy.exe call) is wrapped in quotes, and you might need to swap some single quotes for double quotes.

A reminder – you’ve just told Visual Studio to use a different method of deploying your project. So if strange things suddenly occur when you publish with Visual Studio, be prepared to revert this change.

Deploying Without Visual Studio

The MsDeploy commands are long and complex, so we’re going to wrap them up in MsBuild targets for simplicity.

First, as we go, I’ll be replacing hard-coded values with properties. So in your MsBuild file, inside the “Project” element, add a PropertyGroup that looks like:

<Configuration Condition=" '$(Configuration)' == '' ">Release</Configuration>

This creates a property called “Configuration”, gives it a default value (“Release”), and lets you use it later within a Target by inserting “$(Configuration)” into a command. You can override the default with a “/p:Configuration=Test” argument when running the build. I’ll be using these properties throughout to make the various commands readable and configurable. The full set of properties we need looks like this, either add them all now or as you need them:

<!-- Exes need full path specified if not already available on %PATH% -->
<MSBuildExe Condition=" '$(MSBuildExe)'=='' ">msbuild.exe</MSBuildExe>
<MSDeployExe Condition=" '$(MSDeployExe)'=='' ">msdeploy.exe</MSDeployExe>
<Configuration Condition=" '$(Configuration)' == '' ">Release</Configuration>
<SolutionName Condition=" '$(SolutionName)' == '' ">DeployableWebApp</SolutionName>
<WebApp Condition=" '$(WebApp)' == '' ">DeployableWebApp</WebApp>
<WorkingDir Condition=" '$(WorkingDir)' == '' ">$(MSBuildThisFileDirectory)</WorkingDir>
<PackageAssemblyDir Condition=" '$(PackageAssemblyDir)' == '' ">DeployableWebApp\ReleasePackage</PackageAssemblyDir>
<PackageDeploymentDir Condition=" '$(PackageDeploymentDir)' == '' ">DeploymentPackage</PackageDeploymentDir>
<PublishProfileDeploy Condition=" '$(PublishProfileDeploy)' == '' ">azure-publish-dev</PublishProfileDeploy>
<PublishProfilePackage Condition=" '$(PublishProfilePackage)' == '' ">azure-package</PublishProfilePackage>

<!-- Azure settings from the .PublishSettings file -->
<AzureUsername Condition=" '$(AzureUsername)' == '' ">$john-doe</AzureUsername>
<AzurePassword Condition=" '$(AzurePassword)' == '' "></AzurePassword>
<AzureSite Condition=" '$(AzureSite)' == '' ">site-name</AzureSite>
<AzureUrl Condition=" '$(AzureUrl)' == '' "></AzureUrl>

Something that’s really important when figuring out and testing msdeploy commands is to always start from a “clean” state, so create yourself a “CleanAll” target:

<Target Name="CleanAll">
<RemoveDir Directories="$(binDir);$(ObjDir)"/>
<Exec Command="$(MSBuildExe) $(SolutionName).sln /t:clean"/>

Run it with “msbuild WebDeploy.msbuild /t:CleanAll” and watch it clean.

If you want to just check your code builds as well, create yourself a “Build” target:

<Target Name="Build">
<Exec Command='$(MSBuildExe) $(SolutionName).sln /t:Build /p:Configuration=$(Configuration)'/>

And a quick point about the MsBuild syntax: I’m calling MsBuild from within MsBuild, using the msbuild.exe like you would on the command line. MsBuild also has its own “MsBuild” task syntax, which looks like this:

<MSBuild Projects="$(SolutionName).sln" Targets="Build" Properties="Configuration=$(Configuration)"/>

Use either version, they both call MsBuild with the specified script (a “.sln”, “.csproj”, or your own custom MsBuild file), and run the specified target with the supplied properties.

Building and deploying in a single step

So a quick detour – our goal is to build a package and then deploy that package in a separate step. But first, let’s take the “build and deploy” example above and remove Visual Studio from the process.

We’re going to create a “BuildAndDeploy” target. And to make sure we do a “CleanAll” first, the “DependsOnTargets” lets you specify other targets that must be run first. My initial target looks like this:

<Target Name="BuildAndDeploy" DependsOnTargets="CleanAll">
<Exec Command='$(MSBuildExe) $(WebApp)\$(WebApp).csproj /t:Build /p:VisualStudioVersion=11.0 /p:DeployOnBuild=True /p:PublishProfile="$(PublishProfileDeploy)" /p:Configuration=$(Configuration) /p:Password="$(AzurePassword)"'/>

Note: when setting up my publish profile, I checked “save password”. Inspecting the .pubxml shows a “_SavePWD” property but no “Password” property. Checking the generated msdeploy command shows no password, and deployment fails. So add your password to the publish profile (in “Password” property) and don’t override on your MsBuild call, or add a “/p:AzurePassword=letmein” argument to the msbuild call. And yes, having a property called “AzurePassword” that we then assign to “Password” looks strange. I prefixed all the Azure properties with “Azure” for readability. If we really did want to do a single-step “build and deploy”, using the expected “Password” property would make a lot more sense, but the whole point of this post is for those times when you don’t want to “build and deploy”.

You’ll notice that we’re not calling MsDeploy directly, just building the project, but that triggers calling MsDeploy. You’ll also notice from the msdeploy command that the “.SetParameters.xml” file is referenced to supply some of the deployment values. Also, setting the VisualStudioVersion is probably necessary (removing it can cause issues with directory paths).

Building a deployment package

Having to build the code at deployment time isn’t always very useful, because you end up rebuilding the same thing over and over as you promote the same code to different environments (also, it can screw up your versioning strategy). So we already saw how Visual Studio’s publish does a two-step “build/assemble the site, then deploy it”. We’d like to perform these as separate actions:

    1. Build and assemble into a package, environment-neutral (optionally stamped with a version number)
    2. Some time later deploy the package to test, applying “test” configuration
    3. Some time later deploy the package to production, applying “production” configuration

This is important: The package doesn’t know where it will eventually be deployed and configured.

So in Visual Studio, go to publish your site, but this time create a new publish profile of type “Web Deploy Package”. In settings, for the package location enter “ReleasePackage\” and leave the site name blank. Publish and you get a zip of your site and a few XML files. You’ll also get a “.deploy.cmd” file, which contains everything you need for a one-click deploy. Except, it’s missing a few necessary parameters and values that you need to figure out. Thanks, Microsoft. Actually, all the deployment “.cmd” gives you is the MsDeploy command we’re about to build.

Time to recreate this in our script, so back to the MsBuild, and add some new targets:

<Target Name="BuildPackage" DependsOnTargets="CleanAll">
<Exec Command='$(MSBuildExe) $(WebApp)\$(WebApp).csproj /t:Build /p:VisualStudioVersion=11.0 /p:DeployOnBuild=True;PublishProfile="$(PublishProfilePackage)";ProfileTransformWebConfigEnabled=False;Configuration=$(Configuration)'/>
<CallTarget Targets="_CopyPackageToDeploymentDirectory"/>

<Target Name="_CopyPackageToDeploymentDirectory">
<PackageFiles Include="$(PackageAssemblyDir)\**"/>
<Copy SourceFiles="@(PackageFiles)" DestinationFolder="$(PackageDeploymentDir)\%(RecursiveDir)"></Copy>

So this should build a big ZIP archive of your site, with a few other generated “.cmd” and “.xml” files.

One of the things I’ve added here is the copying of the package from the original “assembly” directory to somewhere else. In real world scenarios, your build will end up generating a package that you then stash somewhere (eg. a “build artefacts” folder) then copy to whatever workspace directory your deployment process runs in for the publishing. So to avoid any reliance on anything other than what’s in the package, you can build the package, move it somewhere for deployment, then clean your workspace.

Note: The MsBuild call for building the package is a little strange – you “build” the project but specify that you want a package (deploy with the publish profile, which creates a package). There’s also a “Package” target, which skips building and compiles a .zip package, but may or may not contain everything. Publishing to a package from Visual Studio gives you a reference zip file.

Deploying the package

Right. Scary experiment time! Add the deployment target:

<Target Name="DeployPackage">
<Exec Command='$(MsDeployExe) -source:package="$(PackageDeploymentDir)\$(WebApp).zip" -dest:auto,ComputerName="https://$(AzureUrl)/msdeploy.axd?site=$(AzureSite)",UserName="$(AzureUsername)",Password="$(AzurePassword)",IncludeAcls="False",AuthType="Basic" -verb:sync -setParam:name="IIS Web Application Name",value="$(AzureSite)" -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -retryAttempts=2 -verbose -userAgent="VS12.0:PublishDialog:WTE12.3.51016.0" -whatif'/>

This is the MsDeploy call we stole from the Visual Studio publish output, tweaked with some readable/configurable parameter values. If msdeploy.exe isn’t already on your path, go to the PropertyGroup and set the value of $(MsDeployExe) to “C:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe” (including the quotes).

OK, try running this:

msbuild WebDeploy.msbuild /t:DeployPackage /p:AzureUsername="U";AzurePassword="P"

(If you haven’t already set the Azure username/password in your build file, make sure you pass the correct values in).

You should see a big long msbuild.exe call in the output, assuming there’s no missing quotes or other typos.

This is the point where it might all fail and spew red text and error messages, and you have to go check the MsDeploy syntax and figure out which parameters to set and what values to use. It’s essential to eliminate anything relating to the “build” environment where the package was created, so ensure no reference to hard-coded build location or deployment Urls in the “package.pubxml” and no use of a generated “parameters.xml” file (a source of many problems).

We haven’t actually published any changes, yet. See the final “-whatif” parameter? That runs MsDeploy in a demo mode where it doesn’t actually change anything (but will complain at the slightest syntax error or missing/incorrect parameter name/value). So remove the “-whatif” and try a real deployment.

Deploying Only What Has Changed

Usually, you only want to deploy the changes. After all, if your application consists of thousands of files, of which only 20 have actually changed in a given build, you want the process to be fast. MsDeploy will deploy only the changes, but by default this is based on file dates. If you do a clean checkout and full rebuild, your “unchanged” files may have a more recent date than their identical counterparts on the server. To prevent redeploying everything, add the “-useCheckSum” argument to your MsDeploy call:

msdeploy.exe -usechecksum

You’ll get a summary of the changes actually shipped when you run MsDeploy, so you can verify if it’s copying too many files.

Bonus: Deploying to IIS

So on my way to figuring out getting MsDeploy to work with Azure, I started with the most basic publish to a local IIS site. If you want to adapt this experiment for IIS instead of Azure, you’ll need the MsDeploy service running on your server.

My original command went something like this (site is named “DeployableWebApp” in IIS):

<Exec Command='$(MsDeployExe) -source:package="" -dest:auto,includeAcls="False" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -setParamFile:"fullpathto.SetParameters.xml"' />
<Exec Command='$(MsDeployExe) -verb:sync -source:contentPath="fullpathtopackage\Web.$(Configuration).config" -dest:contentPath="DeployableWebApp\web.config",includeAcls="False" -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension' />

Note the use of the “SetParameters.xml” file. It didn’t “just work” when I added the step to move the package from an assembly to a deployment directory before deploying. It even complained about the app pool settings (which was probably actually more to do with not finding the dest site and assuming defaults). So let’s get rid of that hassle:

<Exec Command='$(MsDeployExe) -source:package="$(PackageDeploymentDir)\$(WebApp).zip" -dest:auto,includeAcls="False" -verb:sync -setParam:name="IIS Web Application Name",value="DeployableWebApp" -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -retryAttempts=2 -verbose -userAgent="VS12.0:PublishDialog:WTE12.3.51016.0"' />
<Exec Command='$(MsDeployExe) -source:contentPath="$(WorkingDir)$(PackageDeploymentDir)\Web.$(Configuration).config" -dest:contentPath="DeployableWebApp\web.config",includeAcls="False" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -retryAttempts=2 -verbose -userAgent="VS12.0:PublishDialog:WTE12.3.51016.0"' />

Again, the “package” and “selected file” deployments are different. “IIS Web Application Name” (your destination configuration) is not an option when the source is “contentPath”.

Build Once, Deploy Often

If this is somehow working, what you have is pretty awesome – you can call “BuildPackage” then “DeployPackage” for a two-step build-and-deploy process. But we’ve only deployed to a single location. And you only get the default configuration – add a “/p:Configuration=Test” onto the end of your deployment command and nothing happens.

Config transforms and parameters

Managing configuration options, especially when you don’t know at build time which environment(s) your build will get deployed to, is a topic for another post.

However, as a bonus step, it’s worth taking a quick detour into web.configs to illustrate the subtle ways in which MsDeploy can treat shipping deployment packages and individual files differently.

Bonus – Transforming and deploying a web.config

Web.configs aren’t really designed for use in a two-stage “build/package then deploy” process, but you can cheat for now. See this post on Pre-Transforming Web.Configs For Deployment.

Build once, deploy everywhere

What about taking our “dev” package and deploying it to “test”? We need to check that nothing in our generic package is tied to our current dev environment, because we want to deploy the package to “dev”, then “test”, then “production”. We can test this easily:

    1. Go back to your Azure dashboard and create another two web apps for “deployme-test” and “deployme-production”
    2. Grab the publish profile files
    3. Publish!

Now you could go through all the previous steps of creating a publishing profile, testing the build-and-deploy in Visual Studio then the command line, and then trying to publish the package. Or… you could just trust that your “DeployPackage” target actually uses the property values you supply. So as a quick test, you can edit the hard-coded values for your Azure site and credentials, or just skip the properties and create multiple “DeployPackageDev”, DeployPackageTest” etc. targets.

But really, you want to pass these values in on the command-line with the “/p:Param=Value” syntax, and specify your “AzureUrl”, “AzureSite”, “AzureUsername” and “AzurePassword”.

Remember, you have to run “BuildPackage” to get your changes into the package, but then you can deploy that to wherever you want. And after you’ve built the package, you can clean your solution, including the package assembly directory (the package has been copied elsewhere) and deploy that package again and again.

This is where having web.config transforms set up is awesome – you can transform all the configs, bundle them into the package, then have the deployment set whatever configuration works for your target environment.

So you could do

  • “BuildPackage”, “DeployPackage” (with dev settings and config), “DeployPackage” (with test settings and config)
  • “BuildPackage”, “CleanAll”, “DeployPackage” (to dev), “DeployPackage” (to test), etc

One final thing – you may still have your Azure credentials hidden in your build file, which is going to end up in source control with the rest of your project, and maybe even with the package. So you might want to remove these (and pass them in via parameter when you run the deployment script).

Shipping the build script

If you actually want to be deploying these packages you create (eg. from your build server), you’ll need access to the build script. Which probably means shipping the build script with the package (so the correct version of the script is used to create and then ship the package). You can do this with some simple file copying – just create yourself an MsBuild target to have the MsBuild script include itself alongside the package.


The things to look out for

Getting to this point took a lot of trial and error and searching and wondering if it was even possible to drive MsDeploy outside of Visual Studio. The main things that caused me problems:

  • Building the package – beware the project “Package” target, make sure you build and zip up everything (or your deployed app may complain about missing assemblies). Also, the permissions may differ between Visual Studio-generated packages and those your build script spits out (file sizes differ by a few bytes), shouldn’t actually cause any issues.
  • Setting the VisualStudioVersion on build package calls – fail to enforce this and strange things happen, like your published package not ending up in the expected folder, which can then kill subsequent actions (like those web.config transforms). The “DesktopBuildPackageLocation” path in your publish profile may be ignored in favour of just using the “app/obj/Release/Package” directory for assembling the release package.
  • The MsBuild.exe used on the local path may not be the same MsBuild used by your build server, build packaging behaviour differs between versions.
  • Reading the error messages – first make sure you know whether the error is just a typo with your MsBuild/MsDeploy syntax (a misplaced quote), a missing file, or an actual MsDeploy issue.
  • Parameters XML files – caused me endless pain, setting values based on local settings, refusing to let me override values for the remote server, erroneously complaining about ACL errors, AppPool settings (when deploying to IIS), these seem to be a real problem if you don’t publish from the directory you originally assembled the package in. I basically just tried to get rid of these and specify as arguments to my MsDeploy command instead. If you can see everything in one big long MsDeploy command, you might spot issues quicker.
  • Absolute paths and local workspace build paths getting passed to MsDeploy (check those parameters files, and your publish profile). When this ends up on a build server, not only will you be deploying from a different location to where you built your package, but you won’t even be building the package in the same location you did on your local workspace.
  • Relative paths and contentPath – when deploying files (source is “contentPath”) rather than a package, MsDeploy needs a full path to the file, relative paths give you “site does not exist” errors because your source files can’t be found.
  • Viewing a working MsDeploy command – capture the working Visual Studio version, use as a reference, dump the output of your command-line deployment run to a text file and inspect for differences.
  • The MsDeploy “source”/”dest” arguments – take care that you set source to “package” for your zipped package, “contentPath” for individual files, and match with a “dest” of “auto” or “contentPath” (especially if source and dest names change).
  • “IIS Web Application Name” is only applicable in certain deployment cases.
  • The package is not your average “zip” file – if you were to open up the generated .zip package, amend its contents, then zip it up again, you might find deploying it as a package won’t work (complaints about “IIS Web Application Name”, because MsDeploy doesn’t recognise it as a package). Just because a file is called “*.zip”, doesn’t mean it is one. I saw this once when cheating the scripted build process, and I think it’s a compression issue. In these examples, I’m deploying the generated package, then the configs. But you might want to be sneaky and hack the package you’re about to deploy so you can do something clever, like having automatic deployment to a “staging” slot that swaps that to “live” only if the single deployment step is successful.
  • Deploying all the files all the time – add the “useCheckSum” argument to prevent this.

What we have achieved

All this changing the site, doing a “BuildPackage” then a “DeployPackage” is fun, but what have we actually achieved? We should now have the following at our disposal:

  1. A strategy for mimicking Visual Studio’s “build and deploy” process (useful for getting a reference deploy command)
  2. A process for building our web app into an independent deployable package (without using Visual Studio)
  3. A process for deploying our generic web application package to a destination of our choice (without using Visual Studio), we don’t even have to choose between IIS and Azure until we deploy
  4. A process for applying a chosen web.config at deployment time (without using Visual Studio)

And once you replace Visual Studio with something that runs on the command line, suddenly all sorts of automated build and deployment scenarios open up. Imagine a scenario where your build and deployment pipeline is:

  1. Continuous integration builds a release candidate package on every build (and stamps a version number on it)
  2. Package is later deployed to test environment
  3. Later, the same package (that’s the tested code, not the current “head” of your development branch) is deployed to staging, production, etc.

There are a few things missing to get there, and I’ll cover these in other posts:

  1. Configuration – how web.config transforms, and an alternative “deployment parameters” approach, lets you tweak the configuration at deployment time (including shipping to environments that didn’t exist at build time)
  2. Managing the chaos of the MsBuild file when you want generic build and deploy targets to serve multiple apps and environments
  3. Adding these build steps to a build server for true automated deployment
  4. Adding database migrations to the process
  5. Adding other non-web-config-transform configurations (eg. theme assets) to your deployments