"The Magic Tricks of Testing" by Sandi Metz

Earlier this year, Sandi Metz gave a 30-minute talk on how to write good tests at RailsConf in Portland. It was called “The Magic Tricks of Testing.” You can watch the presentation on YouTube here: http://www.youtube.com/watch?v=URSWYvyc42M. The slides are also posted at Speakerdeck: https://speakerdeck.com/skmetz/magic-tricks-of-testing-railsconf.

Seeing as how we are dipping our ignorant feet into the test-driven development waters this week at Flatiron, I figured it wouldn’t hurt to start learning from one of the best. I’m still struggling to see the full value of testing, but Sandi promises in the first few minutes this presentation that testing doesn’t have to suck. AWESOME, I want some of that. Here’s what I learned (plus one super cool reference to Star Trek).

The Main Idea

I’m going to start with her main idea, because it’s one of the greatest things about Ruby: simplicity. She tells us that tests should be an essential tool that we can rely on to write good code. Her talk was specific to unit tests. Wait, what are unit tests? Let’s take a quick detour.

Unit Tests

I was able to infer (and corroborate via StackOverflow) that:

  • Unit tests stand in contrast to integration tests
  • Unit tests focus on small pieces of code, like single objects
  • The intent is to ensure the correct functioning of your app at the atomic level
  • These tests are conducted in isolation, without reference to external objects, as much as possible (using mocks and stubs as placeholders when necessary)

(Integration tests, on the other hand, will test the interaction between your objects)

With our small test suites at Flatiron we have been doing mostly unit testing (I think). This testing model also doesn’t quite apply to the projects we are working on yet, but I think these guiding principles will be good to keep in mind for the near future. Now onto the meat of her presentation.

The Chart

Like any good designer and organized thinker, Sandi used a simple framework to design and explain her unit testing philosophy. It took the form of a chart.

alt text cool and simple chart

This is “the point of this talk,” she says. It’s a great chart once you understand all the pieces. If you’re a pro you probably already know what’s going on. Here’s an explanation for us Rubywans:

The chart applies to a single object (a single unit test suite). Sandi uses an analogy of a space capsule. In a well designed object, similar to a space capsule, nothing solid from outside the object gets in and nothing solid from inside the object gets out — only messages pass through to and from Houston or Moonbase 1 or the Nostromo or whatever. The object receives messages via methods and sends messages via methods. The columns of the chart correspond to the types of messages can send to your object: queries (which return data) and commands (which have side effects). The rows describe the pathway of the messages: incoming (other objects sending signals), sent to self (interior signaling), and outgoing (the object sending signals out to other objects).

The greatest thing about this chart is that HALF OF IT IS BLANK. Three of the six sections say Ignore, in big red letters. That’s key. Tests should make our lives as developers easier. If the tests we are writing make our lives more complicated, we are doing it wrong. Simplicity!

Okay, but why is it blank? We are supposed to be doing testing here.

These messages sent to self aren’t actually the things we care about. Methods that only act on object internals don’t affect other parts of the application. The results of those messages never surface, except through other methods. They are part of the system that generates results for queries and make the state changes directed by commands. Incorrect results will be surfaced by testing from the first and third rows, the incoming and outgoing signal paths.

Testing messages sent to self will also tend to generate anti-patterns. In one case, in order to generate a trigger for the internal methods, you need to contrive a situation to call them, and in doing so you essentially recreate the tests you’ve already done. The other anti-pattern Sandi mentions is overspecification. An overspecification constrains your application too tightly, such that changes you make may not break the application but will break your tests.

Let’s Talk About an Example

Let me demonstrate this principle with an example from a Ruby quiz we recently took at the Flatiron School. It’s a very simple class, but I think it will serve the purpose. The final problem asks us to write a series of tests for the following Path class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Path
  attr_accessor :path

  def initialize(path)
    @path = path
  end

  def normalize_path
    "#{"#{Dir.pwd}/" if relative_path?}#{path}"
  end

  def relative_path?
    !absolute_path?
  end

  def absolute_path?
    path.start_with?("/")
  end
end

simple Ruby class

In the quiz, we needed to write a couple of tests for each method. However, I think Sandi would tell us we don’t need to. The reason? We have four methods, but only two interact with the outside world. The first is the initialize method, which takes a path as an argument. The second is the real workhorse of the class, normalize_path. The other two are just helper methods that help to generate outgoing messages.

Let’s write the tests that we do need. I’m going say argue that with a simple constructor like we have, no test of initialize is necesary. We’d basically be testing that Ruby works. Let’s assume it does. That leaves the normalize_path method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
describe Path#normalize_path do 

    it "returns an absolute a path given an absolute path"
        abs_path = "/Users/joe/my/awesome/path.rb"
        Path.new(abs_path).normalize_path.should eq("/Users/joe/my/awesome/path.rb")

    end

    it "returns an absolute path when given a relative path"
        rel_path = "my/awesome/path.rb"

        Path.new(abs_path).normalize_path.should eq("#{Dir.pwd}/my/awesome/path.rb")
    end

end

two tests for Path#normalize_path

Here’s the Star Trek Reference

This is the logic of the class, in terms of our spaceship metaphor, specifically a Starfleet ship: the Path ship receives a message from Starfleet to normalize the path. The normalize_path method, the captain on the spaceship, calls on relative_path?, the engineering officer on the spaceship. This method in turn calls on the science officer, absolute_path?, who sends some report back to engineering, who interprets that message and in turn sends a more complete report back up to the captain. So far, all the action has been happening inside the spaceship, no transporters required. Furthermore, Starfleet doesn’t want to know about the contents of the reports from the engineering and science officers. We just want to know what the captain has decided. Finally, normalize_path shoots the normalized path back out to Starfleet. The two tests we have written will pass.

The key point here is that if either the absolute_path? method or the relative_path? method was broken, we would receive an incorrect result in our test of normalize_path. If engineering reports incorrect data, then the captain makes the wrong decision. If we can make the two tests that we have pass, we have effectively tested the two internal methods and assured that they work as intended. If we start to write tests for those two, it is a waste of time (although it is good practice for writing tests), because to produce separate tests for the helper methods we will end up recreating the scenario (creating one absolute path and one relative path and matching the outcomes to what we expect to happen). This process is also guilty of overspecification. Once we’ve written the method, we don’t really care how relative_path? works, so if we start mandating that we must have a true or false result, or any specific result at all, we have an artificial constraint. If we want to change the way those methods work in the future we will break these tests, meaning we have to write new tests and double our workload for that change. Our test suite turns into a burden that weighs us down rather than a safety net that frees us to make acrobatic leaps of Ruby.

Conclusion

I have neglected to mention so far any of the many caveats that Sandi applies to her process. Suffice it to say, if it makes more sense to break the rules, then it’s okay to break the rules. In a lot of cases, it is necessary to test complicated internal methods, to ensure that they are sound, but once we know they are working we can delete those tests. Aside from those and a few other exception cases, Sandi tells us that writing good tests is an exercise in minimalism. Most of the time we can do more with less.