5th of Jan 2019: a lot has been happening since I initially wrote this post.
Azure DevOps released a free tier for open source projects, the
GitVersion contributors have been hard at work to take advantage of the latest features of
.NET Core. So much things have changed that I decided to update this post to reflect the current state of affairs (inclusion of
Azure DevOps, upgrade to
.NET Core 2.2, utilisation of
.NET Core global tools and removing the
Mono requirement on
As a developer I’m amazed by the number of free tools and services available. I wanted to create an end-to-end demo of a
CI/CD pipeline that would include:
- Trigger a build on commit
- Use semantic versioning
- Run tests
- Publish test results
- Create NuGet packages
- Publish the NuGet packages
- Create a GitHub release
For my purpose I wanted anonymous users to have access to a read-only view. I initially selected AppVeyor as it seems to be the most popular choice for
.NET open-source projects. But while browsing around I discovered than projects were often using more than one platform. Travis CI and CircleCI seemed to be the two other prevailing options. Since the initial version of this post, Azure DevOps has released a free and unlimited plan for open source projects. I decided to leverage the four platforms so that I could highlight their pros and cons.
The code is hosted on the
GitHub repository Cake build. It’s named Cake after my favourite build automation system and the project is using
Cake as its build system.
Travis CI all use YAML configuration files. This means that your build steps are living in the same space than your code and this presents several benefits:
- Any developer can modify the build
- The project is self-contained
- Developers don’t have to search where the build is located
- It doesn’t matter if something terrible happens to the build server
- Ability to run the build locally on some platforms
I’m sure you’ll be as surprised as I was when I realised how simple the
YAML files are:
Azure DevOps: azure-pipelines.yml
Travis CI: .travis.yml
The project is useless. What is important is that it describes a real-life scenario:
- The solution contains two projects which will be packed as
Logicproject references a
NuGetpackage from nuget.org via a
dotnet packwill turn this into a package reference.
SuperLogicproject depends on
Logicand when packing, this project reference will be turned into a
NuGetpackage reference (handled out of the box by
- The projects target both
net461so they can also be used with the
- The resulting
NuGetpackages should contain
DLLs for both frameworks
- The resulting
- The projects reference a third project that should be embedded as a
DLLrather than being referenced as a
- This is not yet supported by the new tooling but can be achieved.
Mono is not required any more when building on
macOS. This is a massive achievement from the Cake and GitVersion contributors. The build step installing
Travis CI never took less than
5 minutes and would sometimes take over
10 minutes on
Travis CI! As a result the build script has been simplified and is doing less platform specific handling.
Pinning the version of
Cake guarantees you’ll be using the same version of
Cake on your machine and on the build servers. This is enforced via the bootstrap scripts for developers’ machines (bootstrap.ps1 on
Windows, bootstrap.sh on
Unix) and in the
YAML files for the build servers.
As I’m releasing packages I decided to use semantic versioning.
Consider a version format of
Major.Minor.Patch). Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.
Semantic versioning allows the consumers of your binaries to assess the effort to upgrade to a newer version. Semantic versioning should not be used blindly for all kinds of projects. It makes a lot of sense for a
NuGet package but it doesn’t for a product or an
API for example.
.NET we use four properties to handle versioning:
AssemblyInformationalVersionto version assemblies
PackageVersionto version a
Versioning an assembly
StackOverflow are great at explaining how to version an assembly.
AssemblyVersion: the only version the
CLRcares about (if you use strong named assemblies)
Curiously enough the official documentation is sparse on the topic but this what I came up with after doing some reading:
AssemblyVersioncan be defined as
<major-version>.<minor-version>.<build-number>.<revision>where each of the four segment is a
stringthat attaches additional version information to an assembly for informational purposes only. Corresponds to the product’s marketing literature, packaging, or product name
AssemblyInformationalVersion is well documented.
AssemblyFileVersion: intended to uniquely identify a build of the individual assembly
Developers tend to auto-increment this on each build. I prefer it linked to a
tag to be able to reproduce a build. I also use the same
AssemblyFileVersion (I’m a bad person I know).
PackageVersion: A specific package is always referred to using its package identifier and an exact version number
NuGet package versioning is described here.
I’ve implemented semantic versioning using GitVersion. I recommend using GitHub Flow when working on a simple package. In my experience Trunk Based Development tends to lead to lower code quality. Developers push early and often thinking they will fix it later but we all know than in software development later means never.
GitVersion produces an output that will allow you to handle even the trickiest situations:
In my case I’m using:
If you want to handle rebasing and
Pull Requests you’ll have to use a more complex versioning strategy (keep in mind that
GitHub does not support rebasing in
As an aside
Cake allows you to set the
AppVeyor build number.
Run the tests
Travis CI and
CircleCI are running on
macOS they don’t support testing against
net461. Luckily the framework can be enforced using an argument:
Publish the test results
CircleCI has a few quirks when it comes to testing.
First it only support the JUnit format so I had to write a transform to be able to publish the test results. Then you must place your test results within a folder named after the test framework you are using if you want
CircleCI to identify your test framework.
When the tests run successfully
CirceCI will only display the slowest test:
I don’t understand the use case, I would prefer the list of tests and timing and the ability to sort them client-side.
The output for failed tests is not ideal but it might be due to the way I transform the test results:
I decided not to invest more time on this as
CircleCI has zero documentation around publishing test results.
Again, the integration between
AppVeyor shines in this area as
Cake will automatically publish the test results for you (I wondered why I had duplicate test results until I RTFM).
AppVeyor displays all the tests but you must hover to see the framework used:
Failed tests come with a nice formatting and a
Travis CI you may ask. It turns out
Travis CI doesn’t parse test results! All you can rely on is the build log, luckily for us
xUnit is awesome:
.NET Core is leveraging the new
*.csproj system and this means:
- No more
- No more
- No more tears
The references (projects and packages) and the package configuration are both contained in the
*.csproj making it the single source of truth!
Referencing a project without turning it into a package reference
Sometimes you want to include a
DLL in a
NuGet package rather than add it as a package reference.
SuperLogic project depends on the
ExtraLogic project but we don’t want to ship
ExtraLogic as a package. Instead we want to include
Contoso.Hello.ExtraLogic.dll in the
SuperLogic package directly. Currently this is not supported out of the box but the team is tracking it.
Luckily this issue provides a workaround. All the modifications will take place in
- In the
<PropertyGroup>section add the following line:
- Prevent the project to be added as a package reference by making all assets private.
- Finally add the target responsible of copying the
The result is the following
And the assemblies have been versioned as expected:
Note: you can also use the new
*.csproj system for
NuGet packages. You don’t need to target
.NET Core to take advantage of it.
On any branches starting with
NuGet packages will be published to a pre-release feed. If the branch is
master it’ll be published to the production feed. This is handled by
AppVeyor in this section of the configuration.
As this is a demo project both feeds are hosted by
MyGet. For my other projects I use
MyGet to host my pre-release feed and
NuGet for my production feed.
When publishing the packages, I’m also publishing the associated symbols to allow consumers to debug through my packages.
Travis CI does not support artifacts out of the box. You must provide an
S3 account if you wish to save your build artifacts.
Create a GitHub release
AppVeyor also creates
What about Azure DevOps
Azure DevOps is the only product that supports
Microsoft has been iterating relentlessly and the
GitHub acquisition will likely lead to a tighter integration between the two services.
Azure DevOps has the most powerful tests results tab:
One thing I’ve noticed is that builds seem to be slower on the
Hosted Ubuntu 1604 agents than on the
Hosted VS2017 agents.
This is one possible workflow only. I’ve glossed over many details and taken some shortcuts (for example there is no support for
Those are the key takeaways:
- Do upfront planning on how you want to handle versioning. This is the hardest part and the one that will be the hardest to fix later on. Read the GitVersion documentation carefully before making any decision.
- Do what works for your team. If you didn’t have any issues with auto-incrementing your builds, keep doing so. There is no point bringing additional complexity to fix a problem you don’t have.
- Don’t assume you’ll be running on
Visual Studio Enterpriseinstalled. Adding cross-platform or other
Code…) support from the start will make your life easier down the track.