The Art of Unit Testing: with examples in C#

Author: Roy Osherove
4.4
All Stack Overflow 10
This Year Stack Overflow 5
This Month Stack Overflow 1

The Art of Unit Testing: with examples in C#

4.4

Review Date:

Comments

by anonymous   2018-03-19

Let's see at the following statement in Module.ApplicationEndRequest() method:

context.Response.Write(builder.ToString());

When this code is executed from Unit Test, context.Response is a mock that you set up in MoqHttpContext.CreateBaseMocks():

MockResponse = new Mock<HttpResponseBase>();
// ...
MockContext.Setup(ctx => ctx.Response).Returns(MockResponse.Object);

You can't expect that you call a Write() method on a mock and then can read the same data back. Mock is a fake object. Its default implementation of Write() method does nothing, and passed data is just lost.

To fix the problem, you could setup a callback on Response mock that will write passed data to a stream and then return it back on read. You are actually very close to it.

In MoqHttpContext class declare a stream where you will save the data:

public class MoqHttpContext
{
    private readonly MemoryStream _outputStream = new MemoryStream();

    // ...
}

Then in CreateBaseMocks() method setup a callback:

public MoqHttpContext CreateBaseMocks()
{
    // ...

    MockResponse = new Mock<HttpResponseBase>();
    MockResponse.Setup(x => x.Write(It.IsAny<string>())).Callback<string>(s =>
    {
        var data = Encoding.ASCII.GetBytes(s);
        _outputStream.Write(data, 0, data.Length);
        _outputStream.Flush();
        _outputStream.Position = 0;
    });

    // ...
}

You also should remove a line that sets inputStream position to 0 in MoqHttpContextExtensions.StreamWrite(), so that html data that you write in UnitTest1.TestMethod1() is appended, not overwritten:

public static HttpContextBase StreamWrite(this HttpContextBase httpContextBase, Stream inputStream, string text)
{
    if (inputStream == null) inputStream = new MemoryStream();
    var streamWriter = new StreamWriter(inputStream);

    inputStream.Position = inputStream.Length;
    streamWriter.Write(text);
    streamWriter.Flush();
    // Remove this line
    //inputStream.Position = 0;

    return httpContextBase;
}

That's it. Now if you check value of responseRead in the test, you will see that data appended by Http module is there.

UPDATE (Fixing problem with a filter)

There are 3 different issues with current code that prevent correct work of a filter from UT.

  1. You tried handful options for mocking Filter property, however none of them seems correct. The correct way to mock property getter with Moq is:

    MockResponse.SetupGet(response => response.Filter).Returns(filter);
    

    Remove all other statements for mocking response.Filter, but don't add above statement yet, it's not a final version.

  2. You have following check in Module.ApplicationEndRequest() method:

    if (context.Response.Filter is ResponseSniffer filter)
    {
        // ...
    

    When UT is executed, context.Response.Filter is a MemoryStream not a ResponseSniffer. Setter that is called in Module constructor:

    context.Response.Filter = new ResponseSniffer(context.Response.Filter);
    

    will not actually affect value returned by Filter getter because it's a mock that currently always return instance of MemoryStream that you setup with SetupGet. To fix this problem you should actually emulate property behavior: save the value passed to setter and return it in the getter. Here is a final setup of response.Filter property:

    Stream filter = new MemoryStream();
    MockResponse.SetupSet(response => response.Filter = It.IsAny<Stream>()).Callback<Stream>(value => filter = value);
    MockResponse.SetupGet(response => response.Filter).Returns(() => filter);
    

    Make sure you have deleted all other mocks of response.Filter property.

  3. The final problem that you should fix - is the order of Module invocations from UT. Currently the order is the following:

    httpContext.StreamWrite(httpContext.Response.Filter, html);
    // ...
    var module = new Module();
    module.PreRequestHandlerExecute(mockHttpContext.HttpContext());
    

    But PreRequestHandlerExecute sets Response.Filter with an instance of ResponseSniffer. So when httpContext.StreamWrite above it is called, httpContext.Response.Filter holds actually instance of MemoryStream, not ResponseSniffer. So the last fix you should make is to change the order of statements in UT body:

    // ...
    
    var module = new Module();
    module.PreRequestHandlerExecute(mockHttpContext.HttpContext());
    
    httpContext.ResponseWrite(html);
    httpContext.StreamWrite(httpContext.Response.Filter, html);
    
    module.ApplicationBeginRequest(mockHttpContext.HttpContext());
    module.ApplicationEndRequest(mockHttpContext.HttpContext());
    
    // ...
    

UPDATE (UT Redesign)

At this point your UT should work. However current test is very cumbersome. The fact that it takes so much time to understand why it does not work proves it. Such tests are very hard to maintain and fix, they become a real pain over time.

Moreover it's rather Integration test than Unit test, because it invokes several of classes with different functionality - ResponseSniffer and Module.

You should strongly consider redesign of current test. And the good start is to make separate tests for ResponseSniffer and Module classes.

Most valuable test for ResponseSniffer is the one that verifies that written data is registered in RecordStream:

[TestClass]
public class ResponseSnifferTests
{
    [TestMethod]
    public void Write_WritesDataToRecordStream()
    {
        //  Arrange

        var inData = new byte[] { 0x01 };

        var target = new ResponseSniffer(Mock.Of<Stream>());

        //  Act

        target.Write(inData, 0, inData.Length);

        //  Assert

        target.RecordStream.Position = 0;
        var outData = new byte[inData.Length];
        int outSize = target.RecordStream.Read(outData, 0, outData.Length);
        Assert.AreEqual(inData.Length, outSize);
        CollectionAssert.AreEqual(inData, outData);
    }
}

As regards Module class, there are several checks that should be made:

  • PreRequestHandlerExecute() sets Response.Filter with instance of ResponseSniffer.
  • ApplicationBeginRequest() adds Stopwatch to context.Items dictionary.
  • ApplicationEndRequest() writes request info to the response.

UT approach implies checking of these facts in separate tests. Here are samples of such 3 tests:

[TestClass]
public class ModuleTests
{
    [TestMethod]
    public void PreRequestHandlerExecuteShouldSetResponseSnifferAsFilter()
    {
        //  Arrange

        Stream filter = null;
        Mock<HttpResponseBase> httpResponseMock = new Mock<HttpResponseBase>();
        httpResponseMock.SetupSet(response => response.Filter = It.IsAny<Stream>()).Callback<Stream>(value => filter = value);

        Mock<HttpContextBase> httpContextStub = new Mock<HttpContextBase>();
        httpContextStub.SetupGet(x => x.Response).Returns(httpResponseMock.Object);

        var target = new Module();

        //  Act

        target.PreRequestHandlerExecute(httpContextStub.Object);

        //  Assert

        Assert.IsNotNull(filter);
        Assert.IsInstanceOfType(filter, typeof(ResponseSniffer));
    }

    [TestMethod]
    public void ApplicationBeginRequestShouldStoreStopwatchInContextItems()
    {
        //  Arrange

        var items = new Dictionary<string, object>();

        Mock<HttpContextBase> httpContextStub = new Mock<HttpContextBase>();
        httpContextStub.SetupGet(x => x.Items).Returns(items);

        var target = new Module();

        //  Act

        target.ApplicationBeginRequest(httpContextStub.Object);

        //  Assert

        Assert.IsTrue(items.ContainsKey("X-ResponseTime"));
        Assert.IsInstanceOfType(items["X-ResponseTime"], typeof(Stopwatch));
    }

    [TestMethod]
    public void ApplicationEndRequestShouldAddRequestInfoToResponse()
    {
        //  Arrange

        Mock<HttpRequestBase> httpRequestMock = new Mock<HttpRequestBase>();
        httpRequestMock.SetupGet(x => x.FilePath).Returns("/test");

        string writtenData = null;
        Mock<HttpResponseBase> httpResponseMock = new Mock<HttpResponseBase>();
        httpResponseMock.Setup(x => x.Write(It.IsAny<string>())).Callback<string>(s => writtenData = s);

        Mock<HttpContextBase> httpContextStub = new Mock<HttpContextBase>();
        httpContextStub.SetupGet(x => x.Request).Returns(httpRequestMock.Object);
        httpContextStub.SetupGet(x => x.Response).Returns(httpResponseMock.Object);
        httpContextStub.SetupGet(x => x.Items).Returns(new Dictionary<string, object> { ["X-ResponseTime"] = new Stopwatch() });

        var target = new Module();

        //  Act

        target.ApplicationEndRequest(httpContextStub.Object);

        //  Assert

        Assert.IsTrue(Regex.IsMatch(writtenData, @"Response Size: \d+ bytes<br/>"));
        Assert.IsTrue(Regex.IsMatch(writtenData, @"Module request time: \d+ ms"));
    }
}

As you see, the tests are pretty simple and straightforward. You don't need those MoqHttpContext and MoqHttpContextExtensions with a lot of mocks and helpers anymore. Another benefit - if some of the tests get broken, it's much easier to identify the root cause and fix it.

If you are new to Unit Testing and are looking for good source of info on it, I strongly suggest book The Art of Unit Testing by Roy Osherove.

by anonymous   2018-03-19

To illustrate the usage of stubs and mocks, I would like to also include an example based on Roy Osherove's "The Art of Unit Testing".

Imagine, we have a LogAnalyzer application which has the sole functionality of printing logs. It not only needs to talk to a web service, but if the web service throws an error, LogAnalyzer has to log the error to a different external dependency, sending it by email to the web service administrator.

Here’s the logic we’d like to test inside LogAnalyzer:

if(fileName.Length<8)
{
 try
  {
    service.LogError("Filename too short:" + fileName);
  }
 catch (Exception e)
  {
    email.SendEmail("a","subject",e.Message);
  }
}

How do you test that LogAnalyzer calls the email service correctly when the web service throws an exception? Here are the questions we’re faced with:

  • How can we replace the web service?

  • How can we simulate an exception from the web service so that we can test the call to the email service?

  • How will we know that the email service was called correctly or at all?

We can deal with the first two questions by using a stub for the web service. To solve the third problem, we can use a mock object for the email service.

A fake is a generic term that can be used to describe either a stub or a mock.In our test, we’ll have two fakes. One will be the email service mock, which we’ll use to verify that the correct parameters were sent to the email service. The other will be a stub that we’ll use to simulate an exception thrown from the web service. It’s a stub because we won’t be using the web service fake to verify the test result, only to make sure the test runs correctly. The email service is a mock because we’ll assert against it that it was called correctly.

[TestFixture]
public class LogAnalyzer2Tests
{
[Test]
 public void Analyze_WebServiceThrows_SendsEmail()
 {
   StubService stubService = new StubService();
   stubService.ToThrow= new Exception("fake exception");
   MockEmailService mockEmail = new MockEmailService();

   LogAnalyzer2 log = new LogAnalyzer2();
   log.Service = stubService
   log.Email=mockEmail;
   string tooShortFileName="abc.ext";
   log.Analyze(tooShortFileName);

   Assert.AreEqual("a",mockEmail.To); //MOCKING USED
   Assert.AreEqual("fake exception",mockEmail.Body); //MOCKING USED
   Assert.AreEqual("subject",mockEmail.Subject);

 }
}
by anonymous   2018-01-29
@Rogério *"just my extensive real-world experience"* Then it stands against *my* extensive real world experiences as well as of [Robert C "uncle Bob" Martin](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) and [Roy Osherove](https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890) to name two others...
by anonymous   2018-01-14
This boils down to the questioon *what is a `unit`?* I like the answer of *Roy Osherove* in [The Art of Unit Testing](https://www.amazon.com/Art-Unit-Testing-Roy-Osherove/dp/1617290890): *A unit is all the code that has the same reason to change.*
by anonymous   2017-08-20

The standard approach to testing code that runs SQL queries is to unit-test it. (There are higher-level kinds of testing than unit testing, but it sounds like your problem is with a small, specific part of your application so don't worry about higher-level testing yet.) Don't try to test the queries directly, but test the result of the queries. That is, write unit tests for each of the C# methods that runs a query. Each unit test should insert known data into the database, call the method, and assert that it returns the expected result.

The two most common approaches to unit testing in C# are to use the Visual Studio unit test tools or NUnit. How to write unit tests is a big topic. Roy Osherove's "Art of Unit Testing" should be a good place to get started.

by anonymous   2017-08-20

About testability

Due to the use of singletons and static classes MyViewModel isn't testable. Unit testing is about isolation. If you want to unit test some class (for example, MyViewModel) you need to be able to substitute its dependencies by test double (usually stub or mock). This ability comes only when you provide seams in your code. One of the best techniques used to provide seams is Dependency Injection. The best resource for learning DI is this book from Mark Seemann (Dependency Injection in .NET).

You can't easily substitute calls of static members. So if you use many static members then your design isn't perfect.

Of course, you can use unconstrained isolation framework such as Typemock Isolator, JustMock or Microsoft Fakes to fake static method calls but it costs money and it doesn't push you to better design. These frameworks are great for creating test harness for legacy code.

About design

  1. Constructor of MyViewModel is doing too much. Constructors should be simple.
  2. If the dependecy is null then constructor must throw ArgumentNullException but not silently log the error. Throwing an exception is a clear indication that your object isn't usable.

About testing framework

You can use any unit testing framework you like. Even MSTest, but personally I don't recommend it. NUnit and xUnit.net are MUCH better.

Further reading

  1. Mark Seeman - Dependency Injection in .NET
  2. Roy Osherove - The Art of Unit Testing (2nd Edition)
  3. Michael Feathers - Working Effectively with Legacy Code
  4. Gerard Meszaros - xUnit Test Patterns

Sample (using MvvmLight, NUnit and NSubstitute)

public class ViewModel : ViewModelBase
{
    public ViewModel(IMessenger messenger)
    {
        if (messenger == null)
            throw new ArgumentNullException("messenger");

        MessengerInstance = messenger;
    }

    public void SendMessage()
    {
        MessengerInstance.Send(Messages.SomeMessage);
    }
}

public static class Messages
{
    public static readonly string SomeMessage = "SomeMessage";
}

public class ViewModelTests
{
    private static ViewModel CreateViewModel(IMessenger messenger = null)
    {
        return new ViewModel(messenger ?? Substitute.For<IMessenger>());
    }

    [Test]
    public void Constructor_WithNullMessenger_ExpectedThrowsArgumentNullException()
    {
        var exception = Assert.Throws<ArgumentNullException>(() => new ViewModel(null));
        Assert.AreEqual("messenger", exception.ParamName);
    }

    [Test]
    public void SendMessage_ExpectedSendSomeMessageThroughMessenger()
    {
        // Arrange
        var messengerMock = Substitute.For<IMessenger>();
        var viewModel = CreateViewModel(messengerMock);

        // Act
        viewModel.SendMessage();

        // Assert
        messengerMock.Received().Send(Messages.SomeMessage);
    }
}
by anonymous   2017-08-20

but Bar's constructor still depends on Foo, a concrete implementation. I haven't gained anything (have I?).

What you gained here is that when the dependency Foo itself gets any dependencies of its own, or requires a different lifestyle, you can make this change without having to do sweeping changes throughout all consumers of Foo.

I have no need for any other implementations except Foo (test doubles don't count)

You can't just ignore unit testing in this. As Roy Osherove explained a long time ago, your test suite is another (equally important) consumer of your application with its own requirements. If adding the abstraction simplifies testing, you shouldn't need another reason for creating it.

To create an "IFoo" interface or a "FooBase" abstract class surely violates the YAGNI principle?

You won't violate YAGNI if you create this abstraction for testing. In that case YNI (You need it). By not creating the abstraction you you are optimizing locally within your production code. This is a local optimum instead of a global optimum, since this optimization doesn't take all the other (equally important) code that needs to be maintained (i.e. your test code) into consideration.

What is wrong with injecting a concrete implementation since I can refactor this to an abstraction

There isn't anything wrong per see to inject a concrete instance, although -as said- creating an abstraction could simplify testing. If it doesn't simplify testing and letting the consumer take a hard dependency on the implementation could be fine. But do note that depending on a concrete type can have its downsides. For instance, it becomes harder to replace it with a different instance (such as an interceptor or decorator) without having to make changes to the consumer(s). If this is not a problem, you might as well use the concrete type.

by anonymous   2017-08-20

First of all, it is important to understand that Verify-family methods are there for a reason -- they allow you to test unobservable1 behavior of your system. What do I mean by that? Consider simple example of application generating and sending reports. Your final component will most likely look like this:

public void SendReport(DateTime reportDate, ReportType reportType)
{
    var report = generator.GenerateReport(reportDate, reportType);
    var reportAsPlainText = converter.ConvertReportToText(report);
    reportSender.SendEmailToSubscribers(body: reportAsPlainText);
}

How do you test this method? It doesn't return anything, thus you cannot check values. It doesn't change state of the system (like, flipping some flag), thus you cannot check that either. The only visible result of SendReport being called is the fact that report was sent via SendEmailToSubscribers invocation. This is the main responsibility of SendReport method -- and this is what unit tests should verify.

Of course, your unit tests should not and will not check whether some email was sent or delivered. You will verify mock of reportSender. And this is where you use Verify methods. To check that some call to some mock actually took place.

As a final note, Roy Osherove in his book Art Of Unit Testing (2nd edition) separates unit tests into three categories, depending on what can be checked:

  • return value of method (simple, common)
  • change in system state (simple, rare)
  • call to external component (complex, rare)

Last category is where you use mocks and Verify methods on them. For other two, stubs are enough (Setup methods).

When your code is designed correctly, such test will (last category) be in minority in your code base, somewhere in 5% - 10% range (number taken from Roy's book, in line with my observations).


1: Unobservable as in that caller cannot easily verify what exactly happend after the call.

by anonymous   2017-08-20

You're approaching this the wrong way. Just test your functionality: if an exception is thrown the test will automatically fail. If no exception is thrown, your tests will all turn up green.

I have noticed this question garners interest from time to time so I'll expand a little.

Background to unit testing

When you're unit testing it's important to define to yourself what you consider a unit of work. Basically: an extraction of your codebase that may or may not include multiple methods or classes that represents a single piece of functionality.

Or, as defined in The art of Unit Testing, 2nd Edition by Roy Osherove, page 11:

A unit test is an automated piece of code that invokes the unit of work being tested, and then checks some assumptions about a single end result of that unit. A unit test is almost always written using a unit testing framework. It can be written easily and runs quickly. It's trustworthy, readable, and maintainable. It's consistent in its results as long as production code hasn't changed.

What is important to realize is that one unit of work usually isn't just one method but at the very basic level it is one method and after that it is encapsulated by other unit of works.

enter image description here

Ideally you should have a test method for each separate unit of work so you can always immediately view where things are going wrong. In this example there is a basic method called getUserById() which will return a user and there is a total of 3 unit of works.

The first unit of work should test whether or not a valid user is being returned in the case of valid and invalid input.
Any exceptions that are being thrown by the datasource have to be handled here: if no user is present there should be a test that demonstrates that an exception is thrown when the user can't be found. A sample of this could be the IllegalArgumentException which is caught with the @Test(expected = IllegalArgumentException.class) annotation.

Once you have handled all your usecases for this basic unit of work, you move up a level. Here you do exactly the same, but you only handle the exceptions that come from the level right below the current one. This keeps your testing code well structured and allows you to quickly run through the architecture to find where things go wrong, instead of having to hop all over the place.

Handling a tests' valid and faulty input

At this point it should be clear how we're going to handle these exceptions. There are 2 types of input: valid input and faulty input (the input is valid in the strict sense, but it's not correct).

When you work with valid input you're setting the implicit expectancy that whatever test you write, will work.

Such a method call can look like this: existingUserById_ShouldReturn_UserObject. If this method fails (e.g.: an exception is thrown) then you know something went wrong and you can start digging.

By adding another test (nonExistingUserById_ShouldThrow_IllegalArgumentException) that uses the faulty input and expects an exception you can see whether your method does what it is supposed to do with wrong input.

TL;DR

You were trying to do two things in your test: check for valid and faulty input. By splitting this into two method that each do one thing, you will have much clearer tests and a much better overview of where things go wrong.

By keeping the layered unit of works in mind you can also reduce the amount of tests you need for a layer that is higher in the hierarchy because you don't have to account for every thing that might have gone wrong in the lower layers: the layers below the current one are a virtual guarantee that your dependencies work and if something goes wrong, it's in your current layer (assuming the lower layers don't throw any errors themselves).