How to Write Better Automated Tests

Tue Sep 5, 2017 - 1200 Words

As software developers, we should be writing automated tests. Unfortunately, if written poorly tests can eventually become a burden that weighs us down as our projects grow. Today, we’re going to take a look at some things to consider to avoid falling into this situation.

Goals

  • Learn how to think about automated testing.
  • Cover common mistakes made in automated testing.

Note: This is an opinion piece.

Testing is one of those topics that, like text editors, tends to be polarizing. For the sake of this tutorial, we’re going to assume that we all find some value in automated testing and we want to extract as much value as possible out of our tests. To go about getting the most out of testing, we’re going to take a look the mental attitude that can benefit you as you write tests and then look at specific examples of approaches and anti-patterns in testing.

Thinking About Tests

The first big hurdle that people run into when they’re getting started with automated testing are the internal questions of:

  • “Why am I testing this?”
  • “What should I be testing?”
  • “How should I test this?”

The “what” and “why” are pretty important to think about up front as they will inform the “how.”

Why am I Testing This?

For “why” you should be testing you should always look for the benefits that testing can provide. If any of these are something that you value then it’s probably worth it to write automated tests:

  • regression prevention
  • usage informed software design
  • executable documentation / tests as documentation

There are more benefits that you can probably think of, but these are the three main benefits that I hear. If “regression prevention” is your sole reason for writing tests, then might not enjoy writing tests, but you are getting something out of it. The ideas of “usage informed software design” relies on you writing your tests before you implement the code but can provide you with insight into how you’d like the code to be used before you implement it. Lastly, executable documentation can work as a reason to write tests (before or after implementation) if you write your tests in such a way as to show how to use your software.

What Should I Be Testing?

Knowing what to test is something that grows out of experience in writing tests. The more you write tests, the more you gain an idea into when a test is warranted. Here are some simple guidelines that I use when to determine if I should not write a unit test for something:

  • Is the code “private” (not part of my module/object’s public API)? - No explicit tests, if this code is useful, it will be executed by a piece of the public API.
  • Is the code part of another library? - No test. It’s pretty easy to find yourself in a situation where you’re writing a test that only hits code written as part of the framework or library you’re working with. A lot of times this code will be executed by a higher level test, and it’s safe to assume that the library/framework has its own test suite.

Outside of these situations, I tend to write unit tests. Keep in mind, as you move into higher and higher level tests you should likely have fewer tests.

Common Testing Mistakes/Traps

It is incredibly easy to write bad tests. Here are some of the common anti-patterns that I’ve come across and try to avoid. All of the examples will be in Ruby/Rspec, but they will apply to testing in general.

Forgetting to Assert

Depending on your testing framework, your tests may “pass” simply because there was not an assertion made.

require "spec_helper"

RSpec.describe "Failing to assert" do
  it "passes if we never assert" do
    1 == 2
  end
end

This test is entirely wrong, but since there was never an assertion made it will still pass when you run your suite. This is obviously an exceptionally obvious situation, but it’s not that uncommon to forget to add an assertion. A quick way to ensure that you don’t fall into this trap is to make sure that every test can fail. This is where the mantra of “Red, Green, Refactor” really helps. Write the test so that it fails, then make it pass, and finally improve the implementation.

Test Showing the Subject’s Implementation

If you are writing a test thinking about how you’re going to write the implementation or you’ve already written the implementation, then you might fall into the trap of showing the implementation as the assertion.

require "spec_helper"

class AreaExample
  attr_reader :length, :width

  def initialize(length, width)
    @length = length
    @width = width
  end

  def area
    length * width
  end
end

RSpec.describe "Showing Implementation" do
  it "might still be wrong" do
    ex = AreaExample.new(5, 6)
    expect(ex.area).to eql(ex.length * ex.width)
  end
end

This might seem like a decent test, but we’re not making an assertion on the output. We’re expecting the the area to be 30, but because we didn’t explicitly set that value in the assertion, we can break this code without breaking the test. Take a look at how easy it is to remove the value from this test:

require "spec_helper"

class AreaExample
  attr_reader :length, :width

  # omit initialize and area

  def length
    1
  end
end

RSpec.describe "Showing Implementation" do
  it "might still be wrong" do
    ex = AreaExample.new(5, 6)
    expect(ex.area).to eql(ex.length * ex.width)
  end
end

That test will still pass although the return value for area is now 6 instead of 30. Be explicit when writing your tests’ assertions and don’t hint at the implementation of the method.

Stubbing the Object Under Test

Not everyone uses stubs in their tests, but if you do it’s important not to over use them. A good guideline is not to stub the object that you’re testing. Here’s an example where we’re writing tests for a parent class that we expect to be inherited from.

require "spec_helper"

class Worker
  attr_reader :name

  def initialize(name)
    @name = name
  end

  def full_title
    "#{name} #{title}"
  end

  def title
    raise "Must set a worker's title in subclass"
  end
end

RSpec.describe "Overstubbing" do
  it "stubs itself" do
    worker = Worker.new("Kevin Bacon")
    allow(worker).to receive(:title).and_return("PhD.")

    expect(worker.full_title).to eql("Kevin Bacon PhD.")
  end
end

This test will pass, but in reality, this can’t happen since the title method is expecting to be set by a child class. In this scenario, it would be better to assert that the error is raised and then write a useful test for full_title in the tests for the subclass.

Recap

We’ve just covered some ideas and examples around testing that can likely help you write better test suites. Hopefully, these examples give you more clarity as you write tests, but as with any “rules” there are likely situations when you should violate these.