Testing 101 with xUnit

Unit testing your business logic is essential to maintainability. Only if you have a good coverage (read as “as complete as reasonable in your situation”) and enough regression tests (tests proving that something that was broken and got fixed stays fixed) you can be sure not to break anything when refactoring or changing something down the line.

Also if some manager asks you “can your code do X?” and you don’t know: Write a unit test for “X” and you know for now and all time later.

For example I had some piece of code that used a Markdown-to-Markup-rendering library and extended it’s functionality. We had bullet point lists (aka UL tags). My project manager wanted a numbered list (OL tag). So she asked “can your renderer create numbered lists?” I had no idea, if the underlying library could do so, of it I had to make it happen myself. So I built a test, throwing something like

1
2
3

 1. great job
 1. testing
 1. Heureka!

at it and it worked. Now I knew, and I’d also know if it broke for some reason later on. And I could go to my project manager and tell her “yes, it can!”.

So even if you don’t go full TDD (test-driven development), do yourself a favour and test your crucial application parts and at least all explicit requirements.

There are a lot of unit testing frameworks out there like NUnit, MbUnit, MSTest or xUnit. I prefer xUnit for a number of reasons and that’s why I will explain it to you on a fundamental level.

How do I get xUnit?

Visual Studio comes with basic unit testing application templates (also for .NET Core, which can be used for .NET Standard class libs). I like to use the “xUnit Test Project (.NET Core)” template, you can get from right-clicking your solution, choose “Add” -> “New Project” and there go to “Visual C#” -> “.NET Core”.

That will create an empty project with one bare bone test class and all the mandatory dependencies. Just add a reference to your business logic project by right-clicking the “References” node in the Solution Explorer, choose “Add Reference” and check the project to test.

Afterwards you can rename the generated class to what you want and start testing. Of course you need to add a usingstatement to every test class. In our example that would be using Ecommerce.BackOffice.Order.Processing;.

If you want to start by yourself you can add the xunit NuGet package to your dependencies in the test project via right-clicking the project, selecting “Manage NuGet Packages” and install “xunit” from there.

How to structure a solution’s tests

There are different ways you can put your tests into your solution. In fact you could put your unit tests anywhere in your code. But my advice is to properly structure your solution and not to put tests in one project with the business logic it tests. That has a number of reasons, but the most important one is, that you don’t want to deploy your tests to production environments or customer installations.

I found it most useful to have a test project for every project with classes you want to test and a test class within this for every class you want to test. To do this you need to decide on a naming guideline that defines how test projects and classes should be named.

Example: you have a class called Billing in the namespace Ecommerce.BackOffice.Order.Processing and want to test it.

Options for naming the test project would be:

Ecommerce.BackOffice.Order.ProcessingTests
Ecommerce.BackOffice.Order.Processing.Tests
Ecommerce.Tests.BackOffice.Order.Processing
Tests.Ecommerce.BackOffice.Order.Processing

I like adding “Tests” either directly to the project name or as the second token, right after the root namespace, because it feels the most structured to me. The first gives the closest alignment to see what belongs to what and the second clearly separates the tests from the business logic.

The class should be called BillingTests – again for the sake of clarity and structure. Within each test-class testing a single business-class there is a test method for every idea to be tested on any (public) business-class method. It’s a good idea, to build the test method name from the tested method’s name followed by suffixes attributing the character or purpose of the test. Robert C. “Uncle Bob” Martin suggests in his book “Clean Code” to use the situation/input/state explanation as first suffix and the expected outcome as a second, to make it more obvious in your average test tooling. I find that somewhat burdensome, but it is definitely not a bad idea to do so.

Example: The Billing class has a GetTotalPrice method. Possible tests would be GetTotalPrice_Bill_Empty for a bill containing no items GetTotalPrice_Coupon_TotalPercentage to test a bill with a total percentage coupon applied to it. Fully following Uncle Bob’s schema the method name would be like GetTotalPrice_TotalPercentageCoupon_TotalPriceReducedByCouponPercentage.

You might have noticed, that I use underscores here. That is on purpose. Underscores in method names in fact are a violation of Microsoft’s General Naming Conventions for .NET and that – ironically – is why I use them here.

I want to clearly separate what method is tested, what aspect of it is tested and how it is tested without any doubt. If compilers accept underscores in method names and you are not allowed to use them, that makes for a perfect delimiter! What you also can see from the example is, that one business-logic method can be tested from a number of test methods. Usually that is even necessary and strongly advised!

xUnit test method syntax

There are 3 important things to know about the basic syntax of test methods in xUnit:

Test methods are identified by either a [Fact] or a [Theory] attribute.
- If you use [Theory] you also need at least one [InlineData(...)] attribute, where ... stands for valid values for test method parameters.
Every test method has the basic signature of public void METHODNAME(parameters)
- Parameters are only allowed for [Theory] tests, not for [Fact] tests and need to be matched by [InlineData(...)] values.
Every test method needs at least one Assert statement
- Preferably exactly one!

Fact

The Fact attribute defines, that a method depicts exactly one test case. There are no parameters with these, so there is only one scenario to be tested.

Use this if you either want to emphasis this case as something special (e.g. test null-input separate from other invalid inputs) or if a case can not be tested together with similar ones (e.g. having a fact test for a special return of a lookup method for searches without results).

Example:
Testing total price calculation if no items are added to a bill.

1 2	[Fact] public void GetTotalPrice_Bill_Empty()

Theory with InlineData

The Theory attribute defines, that a method depicts a group of tests that belong together. E.g. you could group tests for multiple invalid inputs, that all have the same return, or different valid inputs that get mapped to the same return value.

Sometimes it can even be a good idea to test different input/output pairs, if it is still the same concept that is tested (e.g. testing a parser for different special characters to be escaped properly).

[Theory] always comes together with at least one [InlineData(...)] and every Theory-method has at least one parameter. The values listed in the InlineData brackets have to match the types of the Theory-method and they have to be compile-time constants. So you can stick "a" + "b" in there, but not string.Empty.

Example:

[Theory]
[InlineData("-1", -1)]
[InlineData("1", 1)]
[InlineData("0", 0)]
[InlineData("1234567890", 1234567890)]
public void ParseInt_ValidInputs(string input, int expectedOutput)

Here string gets tested against ints in 4 different cases.

Assert

Any test method needs to assert something. To have clear feedback on the number of fails vs. the number of passes it is a good idea to restrict yourself to a single assert per method. Otherwise if the first assert fails you have no idea, if the rest would pass or not, as a failing assert stops the method’s execution.

Robert “Uncle Bob” Martin, the author of clean code and one of the founders of the TDD movement says in his book “Clean Code” that basically the only time you are allowed to use more than one assertion per test method is if you assert the pre-test state in order for the the post-test assertion to have any meaning or if they share a common concept.

I don’t believe in the latter, but the first. Only do multiple assertions if all but the last only assert, that the result of the last assert can be trusted. So if A fails, the result of B does not matter.

Assert in xUnit is a class with lots of static methods. Each of those static methods reflects one sort of assertion. The ones I use the most are Assert.Equal, Assert.NotEqual, Assert.Null, Assert.NotNull, Assert.True and Assert.False. There is a lot of different possible assertions. All are named with enough common sense, that you can just enter Assert. and look through what IntelliSense offers to find what you need.

You can even assert a specific exception to be thrown. Just call Assert.Throws<NullReferenceException> and hand it a lambda expression like () => billing.GetTotalPrice(). This works for any exception type you like.

Example code

Some example Facts:

using System;
using Ecommerce.BackOffice.Order.Processing;
using Xunit;
 
namespace Ecommerce.BackOffice.Order.ProcessingTests
{
    ///
    /// Tests the Billing class.
    /// 
    public class BillingTests
    {
        ///
        /// Tests correct handling of empty bills.
        /// 
        [Fact]
        public void GetTotalPrice_Bill_Empty()
        {
            Billing billing = new Billing();
            Assert.Equal(0, billing.GetTotalPrice());
        }
 
        ///
        /// Tests correct handling of total percentage Coupons.
        /// 
        [Fact]
        public void GetTotalPrice_Coupon_TotalPercentage()
        {
            Billing billing = new Billing();
 
            // Set a 25% discount coupon.
            billing.Coupon = input =&gt; input * 0.75;
 
            // Add some items.
            billing.Items.Add(new BillingPosition { Name = "Testing", Price = 1234 });
            billing.Items.Add(new BillingPosition { Amount = 0, Name = "Bugfixing", Price = 1234 });
            billing.Items.Add(new BillingPosition { Amount = 10, Name = "New Features", Price = 123 });
 
            Assert.Equal(2464 * 0.75, billing.GetTotalPrice());
        }
 
        ///
        /// Tests correct handling of null-coupons.
        /// 
        [Fact]
        public void GetTotalPrice_Coupon_Null()
        {
            Billing billing = new Billing();
 
            // Set a null coupon.
            billing.Coupon = null;
 
            // Add some items.
            billing.Items.Add(new BillingPosition { Name = "Testing", Price = 1234 });
            billing.Items.Add(new BillingPosition { Amount = 0, Name = "Bugfixing", Price = 1234 });
            billing.Items.Add(new BillingPosition { Amount = 10, Name = "New Features", Price = 123 });
 
            Assert.Throws(() =&gt; billing.GetTotalPrice());
        }
    }
}

These tests are not a clear implementation of the Build-Operate-Check-Pattern (see below), as the operation mostly happens as parameter for the assertion. As long as the operation is simple and has a clear return value and your tests are fairly slim you can get away doing so.

An example Theory:

using Xunit;
 
namespace Ecommerce.BackOffice.Order.ProcessingTests
{
    ///
    /// Tests the official int functionality.
    /// 
    public class IntTests
    {
        ///
        /// Tests if int properly can be parsed from different strings.
        /// 
        ///The input string.
        ///The expected int after parsing.
        [Theory]
        [InlineData("-1", -1)]
        [InlineData("1", 1)]
        [InlineData("0", 0)]
        [InlineData("1234567890", 1234567890)]
        public void TryParse_ValidInputs(string input, int expectedOutput)
        {
            int result = int.Parse(input);
            Assert.Equal(expectedOutput, result);
        }
    }
}

For the full code go to the examples repository on GitHub.

What to test

So now you basically know, how an xUnit test looks like, but we still need to cover what actually needs to be tested in particular.

Which parts of the code

This is a question I struggled with for a long time. It was Jonathan Cutrell, the host of Developer Tea who gave a set of three good rules of thumb for this:

Test anything that seems somewhat reasonable and is public.
Test everything that is critical and public or protected.
If something is private and so critical, that you really want to test it, either test it explicitly through public/protected methods that use it, or maybe it actually wants to be protected.

What do these points mean?

The first rule of thumb refers to public methods. If something is public you should probably test it in some way shape or form. So you look at it closely and see, if it can be tested in a meaningful unit test. If the answer is “yes” you start testing like the next headline describes.

If the answer is “no” there are about 3 reasons for that.

The method is directly working on the UI – most UI logic does not present itself as unit-testable. This might be a job for integration testing, using tools like Selenium.
The method is heavily reliant on or a creator of side effects – this might be a smell. See if you can mock out side effects like network, file system or database access or refactor to a more functional workflow. Side effects generally reduce the testability of your code base and should be kept to a minimum.
The method does more than one thing or is too big – often if you are having trouble testing a method it is too big and does too many things. This is a good point to start breaking up your code and refactor it to smaller chunks.

The second rule of thumb means, if something is accessible (meaning public or protected) and is critical to you or your business it needs to be tested if possible. If it is not possible consider mocking anything out that hinders you.

The third rule of thumb refers to private methods. There might be some mission critical logic in a private method. So critical, that it absolutely has to be tested as a separate unit.

Most of the time private methods get tested indirectly through calls from public or protected methods. And that is fine. If your private method has no influence on any protected or public methods it has no function at all. Any meaningful code can be reached at least indirectly from the outside.

So it is sufficient to test the public/protected API of a class. If the inner signatures or logic change it does not matter, as long as the public interaction and its results are the same. If that does not give you enough peace of mind and it does not create a vulnerability to your code you could actually make the private code protected and test it directly.

Which cases

Every tested concept should be its own method. But what is a concept? What makes a good test case?

It is definitely a good idea to have one test case per type of “happy path“, meaning if every input is as expected and a valid output can be created without a need to fix anything behind the scenes. Maybe your method expects positive integers? Try a Theory with 1, a plausible extremely high number and an average number. If you expect a string, that represents a timestamp create a Theory with American, British, French and German time formats, if all of these should be supported.

This also includes different capabilities a single method has to have. If writing a markdown parser it would be good to have unit test methods for each type of construct (UL, OL, bold, headline…).

Test all obvious wrong input types individually. The int-method could fail for negative numbers and for 0 differently – test those individually. If int.MaxValue is a problem test that as well.

There are also positive and negative edge cases. If parsing dates what do you do with a date like 12/11/09? Even if that input is acceptable a unit test can make for some good documentation of intent. If you default to a specific culture’s interpretation a test can verify and document that. If your method does not accept ambiguous inputs you can verify null return or an Exception here, also documenting intent.

This kind of test also can provide nice regression tests. You app failed in production if a methods gets fed prime integers greater than 1024? Write a test for that case, fix it and be sure that edge won’t cut you again.

How to structure the test method

A very popular pattern for structuring the inner workings of a test method is the Build-Operate-Check-Pattern (as seen in Clean Code – again). The three steps to this pattern are:

Build the state that is the basis for what you plan to test. This could be things like instantiating the class that you are testing and setting properties. Everything you need to do, before you can do, what you want to test. Be sure that everything that is done in here is covered by other tests, so you can trust these actions.

If you have reoccurring preparations it is a common practise to extract those steps to separate methods to reuse them. The builder pattern may be of great use if you plan a big set of tests with complex state.

Operate on the state. Here you do what you actually want to test.

Check that what you did in the operate step yielded the expected results. You do this by using asserts to compare your expectation with the actual results.

Testing your code F.I.R.S.T.

Another good practice are the F.I.R.S.T. rules.

Your tests should be fast. That means they should run quickly.

Your tests should be independent. They should not be dependent on each other or on the order of execution. If done properly, they even can run in parallel without any influence on each other. That also means that no external state is required or changed for any test.

This one bit me once when testing a single class with different states in a static field for different tests. I fixed this by introducing an additional optionaL parameter for the field and only fall back to the static field if left blank. Explicitly setting the parameter in the tests allowed to run the tests in parallel again.

Your tests should be repeatable. Any environment capable of running your tests has to yield the same results. No special infrastructure or other external system should be needed. The only exception I would make for that is if you want to test the interaction with external systems.

One creed for this is “don’t mock what you don’t own”. If there is a version of an external service or can hit without consequences hit the real thing. Otherwise you won’t know if it gets changed and your compatibility breaks.

Your tests should be self-validating. It has to be absolutely clear to you for every test, if it is passing or failing and that should be clear from the assertion alone. No extra state checking or reading logs should be necessary. This is another reason to only have one relevant assertion per test method; there should be no doubt about the full result of any test.

Your tests should be written timely. TDD expects you to write tests as soon as a functional expectation arises. This means before the business logic is written; ideally right before doing so. This makes sure that your tests are easy to code against and your business logic is easily testable. If you’re not doing TDD, still write tests sooner rather than later – it only gets harder over time.

Writing testable code

How do you write your code so it is easy to test and how to you access it from the tests?

As mentioned above it is crucial to write your code in a testable fashion.
This means you keep your separations of concerns clear – have every method do one thing.
This means also to have your methods short and concise.
And this means any external dependencies should be either separated or dependency-injected, so they can be mocked if needed.

public

Public methods are usually fairly easy to test, if you adhered to the ideals above. Static methods can be called directly, non-static methods can be called directly from created instances.

protected

Protected methods can be tested in a similar fashion. The only difference is, that you need a wrapper class that is derived from the tested class to make the protected methods public by accessing the protected method in a new public one.

private

These are the tricky ones. This is where code coverage tools get very useful. To test private methods thoroughly you have to rely on public and protected methods that call it. To see if you caught everything you can use the line-by-line coverage tools of most advanced testing suites like the one coming with ReSharper.

If something cannot be tested thoroughly while being private, you can either cut the code out or should consider making the logic protected instead.

Benefits

Why should you be writing thorough test? There are lots of reasons.

It gives peace of mind, increases confidence in your code and gives early feedback if you break some thing after the fact.

Also it increases the maintainability. If you have testable code you can easily change or add logic without widespread effects and fix bugs where they appear without side effects.

This enables you to react quickly if something breaks in production.

Testable code is also easier to read. It usually speaks for itself and if it does not, the test code is a good documentation of the factual meaning of it.

Testable code a good foundation for a stable architecture. Concise code that is structured in small classes with short methods and clear (injected) dependencies without implicit requirements can be organised in almost every architecture pattern..

Fewer bugs! Most of the bugs that would bite you can be avoided by proper testing.