The scourge of the test context6 min read

Posted on by

I spend a fair amount of time reviewing automation code and frameworks and recently I’ve come to the conclusion that the testing community has a problem. We have an unhealthy obsession with test contexts!

I’m not fundamentally opposed to the use of a context within an automation framework, but I am against some of the bad practices I come across with distressing regularity.

In this post, I will describe some of the common pitfalls I’ve encountered and some simple solutions that will help to speed up your tests and improve the maintainability of your frameworks.

Static Context

The most common mistake I see is the use of static test context objects. Having a static context allows data to be shared between classes and easily accessed from anywhere within a test framework. This can be particularly useful when using test frameworks like SpecFlow which perform the object instantiation for you. The problem with a static context, however, is that it will prevent your tests from executing in parallel. If an object is static there is only one instance. This means that each of your tests can make updates to it simultaneously, resulting in unpredictable race conditions and fragile tests.

 
 
[When(@"the request is sent to the profile Api")]
public void WhenTheRequestIsSentToTheProfileApi()
{
  var request = (string)ScenarioContext.Current["ApiRequest"];
  ScenarioContext.Current["ApiResponse"] = profileApiService.Send(request);
}

From my experience, testers tend to rely on a static context either out of ignorance or simple laziness.

If you’re guilty of this bad practice then don’t despair, there is a solution. The school of code craftsmanship figured this one out a long time ago with the SOLID design principles. The D of SOLID stands for Dependency Injection. For us, what this really means is simply passing data into our classes as parameters, rather than using static references. In terms of a test context, the simple solution is usually to new up an instance of your context at the start of your test and then “inject” it into your classes as a constructor parameter.

 
 
[TestFixture]
public class Tests
{
  private TestSteps testSteps;
  [SetUp]
  public Tests()
  {
    var myTestContext = new MyTestContext();
    testSteps = new TestSteps(myTestContext);
  }
}
public class TestSteps
{
  private readonly MyTestContext myTestContext;
  public TestClass(MyTestContext myTestContext)
  {
    this.myTestContext = myTestContext;
  }
}

It really is that simple.

The complexity comes when you are using a framework such as SpecFlow, that does the instantiation for you, by virtue of the [Binding] attribute. Fortunately, SpecFlow provides a simple workaround – Context Injection.

As long as your context class provides a parameter-less constructor, SpecFlow will automatically create a new instance per teat and inject it into the constructor of your [Binding] classes. Magic!

With this simple technique, you can avoid using static references and enable your tests to run in parallel!

Context Obsession

The second most common coding smell I encounter, with regard to test contexts, is overuse. You know you have this problem if there are references to your context object littered everywhere throughout your test framework. This bad practice often goes hand-in-hand with a static test context but results in its own set of challenges.

When your context is being written to and read from in lots of places or deep inside helper classes and extension methods, the code becomes very difficult to follow and even harder to debug.

The solution to this problem is perhaps less obvious and less definitive. My approach is to apply a simple rule of thumb – only allow step methods (referred to as Tasks in the Screenplay Pattern) to access the context. This means that you only ever have to go to one level of abstraction, to understand the life cycle of the context. When combined with dependency injection (described above), you only then need to inject the context into your first level of classes.

When data does need to be used in downstream classes, these can be retrieved from the context and passed as parameters into those methods. The results of these methods can be returned to the calling classes and then stored back in the context, if necessary.

The other rule is to only store data in your context that truly needs to be shared between classes. There is no benefit in using the test context when a local variable or field will suffice. It simply becomes a maintenance overhead and confusing to anyone that is new to the framework. Be frugal, not frivolous with your context!

The Mystery Box

The third and final bad practice is using a multi-purpose dictionary or generic repository to store the data within the context. This may seem like an elegant and flexible method of storing a variety of data but often turns into an unwieldy mess.

The first problem with a dictionary is that you don’t know what’s in it until you look. This means that when you attempt to read from it, you first have to check that the item you’re looking for is actually there. This can result in lots of error checking to handle for the cases where the thing you’re looking for is missing.

 
 
public static IDictionary<string, string> Tokens
{
  get
  {
    if (ScenarioContext.Current.TryGetValue("Tokens", out var tokensObj) && tokensObj is IDictionary<string, string> tokens)
    {
      return tokens;
    }
    tokens = new Dictionary<string, string>();
    ScenarioContext.Current["Tokens"] = tokens;
    return tokens;
  }
}

The other issue is that you need to manage a set of key values to reference each stored item. This often results in strings being duplicate across the codebase, which can cause headaches when something is misspelt or renamed.

 
 
public static string Request
{
  get => (string)ScenarioContext.Current["Requset"];
  set => ScenarioContext.Current["Request"] = value;
}

The solution to this smell is the simplest of the lot. My approach is to replace the dictionary with strongly typed properties.

This approach makes it absolutely transparent what the context can store and avoids any confusion with literal key values.

I also favour storing full objects (POCOs) over primitives – we are writing object-oriented code after all. As an example, I would usually store a whole customer object, rather than storing the name, email address and id as separate properties / public fields.

 
 
public class MyTestContext
{
  public CustomerRequest Request { get; set; }
  public CustomerResponse Response { get; set; }
}

Summary

Fundamentally nothing I’ve mentioned in this post is remotely revolutionary. The solutions are just good coding practices that can be applied to almost any piece of object-oriented code.

If you’re interested in learning about other code smells and techniques that can be used to avoid them, then take a look at https://sourcemaking.com/refactoring/smells and propel your coding to the next level.

Share this post: