xUnit Test Patterns: Refactoring Test Code

Unit testing is one thing, another thing to consider is test driven development, where the act of writing the tests first affects the design/ feel of the finally delivered code - hopefully for the better. I find this helps especially if the problem domain is not fully understood at the start of programming.

Clarke Ching does a good one hour talk about TDD using excel. If you spend an hour reading through this, you should get the idea.

http://www.clarkeching.com/files/tdd_for_managers_and_nonprogrammers_using_excell_and_vba_final.pdf

You know you have arrived with unit testing when xUnit Test Patterns is an enjoyable read. http://www.amazon.co.uk/xUnit-Test-Patterns-Refactoring-Signature/dp/0131495054/ref=sr_1_1?ie=UTF8&qid=1288638075&sr=8-1

That is probably a big ask initially though, and I would suggest something thinner about either refactoring or TDD would be a more gentle introduction to this fascinating subject.

The case you brought in this question looks a lot like a good candidate for data-driven testing strategy. "xUnit Test Patterns: Refactoring Test Code" by Gerard Meszaros, defines the conditions for a test to qualify for data-driven testing strategy as follows (p 288):

For example, we might want to run essentially the same test with slightly different system inputs and verify that the actual output varies accordingly. Each of these tests would consist of exactly the same steps.

Wikipedia entry on data-driven testing also states:

Anything that has a potential to change (also called "variability," and includes elements such as environment, end points, test data, locations, etc.) is separated out from the test logic (scripts) and moved into an 'external asset'. This can be a configuration or test dataset. The logic executed in the script is dictated by the data values.

The recommended implementation strategy for data-driven testing is for the data to be separated from testing logic. The input and expected output data for each test are stored together in a file or a database and keyed on the test. The testing logic is organized in a modular fashion so as to increase reuse. With the proper implementation language/tools, the testing logic can be grown into a DSL (Domain-Specific-Language) or even a fully-fledged interpreter (recommended in xUnit Test Patterns p288). A consequence of this: the formulation of your tests will be essentially declarative and explicit on the intent and the features being exercised, which will also transform your tests into another meaningful documentation source for your system.

Further reading:

• Section on data-driven testing in xUnit patterns available online.
• Data-driven test automation frameworks: an article with interesting insights on building data-driven test frameworks.

For TDD try xUnit Test Patterns.

The best book I have read recently for filling in the gaps is C# in depth by Jon Skeet. It wasn't until reading this book that I realised how many gaps I had in my knowledge of C#.

The AutoMoq Glue Library gives AutoFixture the additional capabilities of an Auto-Mocking Container; that is, not only can it compose normal objects for you - it can also supply mock objects to your objects, should they be so required.

AutoFixture itself is not an Auto-Mocking Container, but rather a library that automates the Fixture Setup phase of the Four Phase Test pattern, as described in xUnit Test Patterns. It also enables you to automate code associated with the Test Data Builder pattern.

It was explicitly designed as an alternative to Implicit Setup, so I think it makes little sense to use it together with a setup method and mutable class field as this question suggests. In other words, the whole point of AutoFixture is that it enables you to write independent unit tests like this:

[TestFixture]
public class MyFixture
{
    [Test]
    public void MyTest()
    {
        var fixture = new Fixture().Customize(new AutoMoqCustomization());
        var product = fixture.Create<Product>();

        // Use product, and whatever else fixture creates, in the test
    }
}

You definitely can use it to test your units even when they have no dependencies, but in that case, you probably don't need the AutoMoq Customization.

The "InternalsVisibleTo" attribute is key to any sort of "white-box" (the term of the decade, I guess) testing for .Net. It can be placed in any c# file with the "assembly" attribute on the front. Note that MS DOCs say that the assembly name must be qualified by the public key token, if it is signed. Sometimes that does not work and one must use the full public key in it's place. Access to internals is key to testing concurrent systems and in many other situations. See https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054. In this book, Meszaros describes a variety of coding styles that basically constitute a "Design For Test" approach to program development. At least that's the way I've used it over the years.

Besides the xUnit-1.9.1.chm-File mentioned by Sean U and the Examples on the official xUnit.NET website I found two other resources to help me understand the basics of the work with xUnit.NET:

• An Introduction to xUnit.net for ASP.NET MVC Developers by Stephen Walter
• Learning to Use xUnit.Net (3 Chapter Tutorial) by Maria Marcano

Sadly, as pointed out also by Sean U, it seems as there are no books at all about the xUnit.NET-Framework yet. So, for further information it looks like one has go with studying the *.chm-File and reading general books about unit testing. Or switch to another testing-framework, that's what I think I'll do...

Update

Ognyan Dimitrov added some additional resources in his comments:

• XUnitPatterns.com by Gerard Meszaros (Website)
• xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros (Book)

Update 02/2017:

There also seems to be a book now:

• xUnit Test Patterns: Refactoring Test Code

Other hanselminutes episodes on testing:

• #112 The Past, Present and Future of .NET Unit Testing Frameworks - listened a while back, remember being slightly underwhelmed, but still worth a listen
• #103 Quetzal Bradley on Testing after Unit Tests - extremely interesting, giving deep insight into how to think about the purpose of coverage metrics etc.
• #146 Test Driven Development is Design - The Last Word on TDD (with Scott Bellware) - lives up to its name in that it slams home a lot of core concepts that you "always knew anyway". IIRC the podcast gives a favourable mention to the Newkirk/Vorontsov book - which I wouldnt particularly second (it's a long time since I read it -- I might just not have been ready to absorb its messages)

Other podcasts:

• Herding Code Episode 42: Scott Bellware on BDD and Lean Development - recorded after Hanselminutes #146. Again, very good discussion that helps to cement ideas around "classic tests" vs BDD vs Context Specification and various such other attempts at classifications...
• J.B. Rainsberger: "Integration Tests Are A Scam" is a recording of a presentation covering integration vs unit tests.

Other questions like this:

• Good QA / Testing Podcast, which among others lists the meta podcast http://testingpodcast.com/

Blog posts:

• It’s Not TDD, It’s Design By Example - Brad Wilson, similar in vein to HC #42, attempting to get across why you're writing the tests.

I know you didn't ask for books but... Can I also mention that Beck's TDD book is a must read, even though it may seem like a dated beginner book on first flick through (and Working Effectively with Legacy Code by Michael C. Feathers of course is the bible). Also, I'd append Martin(& Martin)'s Agile Principles, Patterns & Techniques as really helping in this regard. In this space (concise/distilled info on testing) also is the excellent Foundations of programming ebook. Goob books on testing I've read are The Art of Unit Testing and xUnit Test Patterns. The latter is an important antidote to the first as it is much more measured than Roy's book is very opinionated and offers a lot of unqualified 'facts' without properly going through the various options. Definitely recommend reading both books though. AOUT is very readable and gets you thinking, though it chooses specific [debatable] technologies; xUTP is in depth and neutral and really helps solidify your understanding. I read Pragmatic Unit Testing in C# with NUnit afterwards. It's good and balanced though slightly dated (it mentions RhinoMocks as a sidebar and doesnt mention Moq) - even if nothing is actually incorrect. An updated version of it would be a hands-down recommendation.

More recently I've re-read the Feathers book, which is timeless to a degree and covers important ground. However it's a more 'how, for 50 different wheres' in nature. It's definitely a must read though.

Most recently, I'm reading the excellent Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce. I can't recommend it highly enough - it really ties everything together from big to small in terms of where TDD fits, and various levels of testing within a software architecture. While I'm throwing the kitchen sink in, Evans's DDD book is important too in terms of seeing the value of building things incrementally with maniacal refactoring in order to end up in a better place.

About testability

Due to the use of singletons and static classes MyViewModel isn't testable. Unit testing is about isolation. If you want to unit test some class (for example, MyViewModel) you need to be able to substitute its dependencies by test double (usually stub or mock). This ability comes only when you provide seams in your code. One of the best techniques used to provide seams is Dependency Injection. The best resource for learning DI is this book from Mark Seemann (Dependency Injection in .NET).

You can't easily substitute calls of static members. So if you use many static members then your design isn't perfect.

Of course, you can use unconstrained isolation framework such as Typemock Isolator, JustMock or Microsoft Fakes to fake static method calls but it costs money and it doesn't push you to better design. These frameworks are great for creating test harness for legacy code.

About design

1. Constructor of MyViewModel is doing too much. Constructors should be simple.
2. If the dependecy is null then constructor must throw ArgumentNullException but not silently log the error. Throwing an exception is a clear indication that your object isn't usable.

About testing framework

You can use any unit testing framework you like. Even MSTest, but personally I don't recommend it. NUnit and xUnit.net are MUCH better.

Further reading

1. Mark Seeman - Dependency Injection in .NET
2. Roy Osherove - The Art of Unit Testing (2nd Edition)
3. Michael Feathers - Working Effectively with Legacy Code
4. Gerard Meszaros - xUnit Test Patterns

Sample (using MvvmLight, NUnit and NSubstitute)

public class ViewModel : ViewModelBase
{
    public ViewModel(IMessenger messenger)
    {
        if (messenger == null)
            throw new ArgumentNullException("messenger");

        MessengerInstance = messenger;
    }

    public void SendMessage()
    {
        MessengerInstance.Send(Messages.SomeMessage);
    }
}

public static class Messages
{
    public static readonly string SomeMessage = "SomeMessage";
}

public class ViewModelTests
{
    private static ViewModel CreateViewModel(IMessenger messenger = null)
    {
        return new ViewModel(messenger ?? Substitute.For<IMessenger>());
    }

    [Test]
    public void Constructor_WithNullMessenger_ExpectedThrowsArgumentNullException()
    {
        var exception = Assert.Throws<ArgumentNullException>(() => new ViewModel(null));
        Assert.AreEqual("messenger", exception.ParamName);
    }

    [Test]
    public void SendMessage_ExpectedSendSomeMessageThroughMessenger()
    {
        // Arrange
        var messengerMock = Substitute.For<IMessenger>();
        var viewModel = CreateViewModel(messengerMock);

        // Act
        viewModel.SendMessage();

        // Assert
        messengerMock.Received().Send(Messages.SomeMessage);
    }
}

What you are looking for is what in xUnit Test Patterns is called Test-Specific Equality.

While you can sometimes choose to override the Equals method, this may lead to Equality Pollution because the implementation you need to the test may not be the correct one for the type in general.

For example, Domain-Driven Design distinguishes between Entities and Value Objects, and those have vastly different equality semantics.

When this is the case, you can write a custom comparison for the type in question.

If you get tired doing this, AutoFixture's Likeness class offers general-purpose Test-Specific Equality. With your Student class, this would allow you to write a test like this:

[TestMethod]
public void VerifyThatStudentAreEqual()
{
    Student st1 = new Student();
    st1.ID = 20;
    st1.Name = "ligaoren";

    Student st2 = new Student();
    st2.ID = 20;
    st2.Name = "ligaoren";

    var expectedStudent = new Likeness<Student, Student>(st1);

    Assert.AreEqual(expectedStudent, st2);
}

This doesn't require you to override Equals on Student.

Likeness performs a semantic comparison, so it can also compare two different types as long as they are semantically similar.

Basically, a Mock is not the real thing, so it does not have real behavior. It's not supposed to have real behavior - it's supposed to do whatever you tell it while keeping track of what happened. Nothing more and nothing less.

This means that you have to tell it how its Copy method works. If you do the following, that's the implmementation the Copy method will have:

Mock<IDataSet> dataMock = new Mock<IDataSet>();
Mock<IDataSet> copyMock = new Mock<IDataSet>();

dataMock.Setup(c => c.Copy()).Returns(copyMock.Object);

However, you can also do this:

Mock<IDataSet> dataMock = new Mock<IDataSet>();
Mock<IDataSet> copyMock = dataMock;

dataMock.Setup(c => c.Copy()).Returns(copyMock.Object);

and that, then, becomes the implementation of the Copy method. Remember: an interface is not a contract that states what the method should do; it only defines the signature of methods.

You were probably hoping to copy data from one IDataSet to another, but remember that a Mock is pure behavior; it has no data.

A couple of alternatives you can think about are the following:

• Replace IDataSet with an abstract DataSetBase class, and implement the Copy method like you want it to behave (that is, not as an abstract or virtual method).
• Instead of creating a Mock of IDataSet, use a Fake. A Fake is a a test-specifict implementation of an interface that has behavior close to the real thing. There are no frameworks/libraries for creating Fakes, so you would need to code such a Fake by hand.
• Consider whether the Copy method should really be part of the interface. It sounds to me like it's an implementation detail that doesn't belong on the interface in the first place.

You can read about Stubs, Mocks, Fakes and other unit testing design patterns in the excellent book xUnit Test Patterns.

To address both of your questions:

1. In both common unit test patterns (Arrange/Act/Assert, and Four-Phase Test), there will almost always be more Arrange code than Act code, since by definition, Act should only contain a single statement. However, it is still a good idea to attempt to minimize the Arrange code as much as possible.

2. There are more than one way to refactor Setup code into reusable code. As Alex writes in his answer, many unit testing frameworks support setup methods. This is called Implicit Fixture Setup, and in my opinion something that should be avoided, since it does not communicate intent very well. Instead, I prefer explicit setup methods, usually encapsulated into a Fixture Object.

In general, the need for complex Setup code should always cause you to consider if you can model your API differently. This is not always the case, but when it is, you often end up with a better and more concise API than you started out with. That's one of the advantages of TDD.

For the cases where setup is just complex because input is complex, I will recommend AutoFixture, which is a general-purpose Test Data Builder.

Many of the patterns I have used in this answer are described in xUnit Test Patterns, which is an excellent book.

I know that Kent Beck's book (which you mentioned) covers TDD in Python to some pretty good depth. If I remember correctly, the last half of the book takes you through development of a unit test framework in Python. There's nothing specific to web development, though, which is a problem in many TDD resources that I've read. It's a best practice to keep your business logic separate from your presentation in order to make your BL more testable, among other reasons.

Another good book that you might want to look into is xUnit Test Patterns. It doesn't use Python, but it does talk a lot about designing for testability, how to use mocks and stubs (which you'll need for testing web applications), and automating testing. It's more advanced than Beck's book, which makes it a good follow-up.