"The Magic Tricks of Testing" by Sandi Metz
Earlier this year, Sandi Metz gave a 30-minute talk on how to write good tests at RailsConf in Portland. It was called “The Magic Tricks of Testing.” You can watch the presentation on YouTube here: http://www.youtube.com/watch?v=URSWYvyc42M. The slides are also posted at Speakerdeck: https://speakerdeck.com/skmetz/magic-tricks-of-testing-railsconf.
Seeing as how we are dipping our ignorant feet into the test-driven development waters this week at Flatiron, I figured it wouldn’t hurt to start learning from one of the best. I’m still struggling to see the full value of testing, but Sandi promises in the first few minutes this presentation that testing doesn’t have to suck. AWESOME, I want some of that. Here’s what I learned (plus one super cool reference to Star Trek).
The Main Idea
I’m going to start with her main idea, because it’s one of the greatest things about Ruby: simplicity. She tells us that tests should be an essential tool that we can rely on to write good code. Her talk was specific to unit tests. Wait, what are unit tests? Let’s take a quick detour.
I was able to infer (and corroborate via StackOverflow) that:
- Unit tests stand in contrast to integration tests
- Unit tests focus on small pieces of code, like single objects
- The intent is to ensure the correct functioning of your app at the atomic level
- These tests are conducted in isolation, without reference to external objects, as much as possible (using mocks and stubs as placeholders when necessary)
(Integration tests, on the other hand, will test the interaction between your objects)
With our small test suites at Flatiron we have been doing mostly unit testing (I think). This testing model also doesn’t quite apply to the projects we are working on yet, but I think these guiding principles will be good to keep in mind for the near future. Now onto the meat of her presentation.
Like any good designer and organized thinker, Sandi used a simple framework to design and explain her unit testing philosophy. It took the form of a chart.
This is “the point of this talk,” she says. It’s a great chart once you understand all the pieces. If you’re a pro you probably already know what’s going on. Here’s an explanation for us Rubywans:
The chart applies to a single object (a single unit test suite). Sandi uses an analogy of a space capsule. In a well designed object, similar to a space capsule, nothing solid from outside the object gets in and nothing solid from inside the object gets out — only messages pass through to and from Houston or Moonbase 1 or the Nostromo or whatever. The object receives messages via methods and sends messages via methods. The columns of the chart correspond to the types of messages can send to your object: queries (which return data) and commands (which have side effects). The rows describe the pathway of the messages: incoming (other objects sending signals), sent to self (interior signaling), and outgoing (the object sending signals out to other objects).
The greatest thing about this chart is that HALF OF IT IS BLANK. Three of the six sections say Ignore, in big red letters. That’s key. Tests should make our lives as developers easier. If the tests we are writing make our lives more complicated, we are doing it wrong. Simplicity!
Okay, but why is it blank? We are supposed to be doing testing here.
These messages sent to self aren’t actually the things we care about. Methods that only act on object internals don’t affect other parts of the application. The results of those messages never surface, except through other methods. They are part of the system that generates results for queries and make the state changes directed by commands. Incorrect results will be surfaced by testing from the first and third rows, the incoming and outgoing signal paths.
Testing messages sent to self will also tend to generate anti-patterns. In one case, in order to generate a trigger for the internal methods, you need to contrive a situation to call them, and in doing so you essentially recreate the tests you’ve already done. The other anti-pattern Sandi mentions is overspecification. An overspecification constrains your application too tightly, such that changes you make may not break the application but will break your tests.
Let’s Talk About an Example
Let me demonstrate this principle with an example from a Ruby quiz we recently took at the Flatiron School. It’s a very simple class, but I think it will serve the purpose. The final problem asks us to write a series of tests for the following
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
In the quiz, we needed to write a couple of tests for each method. However, I think Sandi would tell us we don’t need to. The reason? We have four methods, but only two interact with the outside world. The first is the initialize method, which takes a path as an argument. The second is the real workhorse of the class,
normalize_path. The other two are just helper methods that help to generate outgoing messages.
Let’s write the tests that we do need. I’m going say argue that with a simple constructor like we have, no test of
initialize is necesary. We’d basically be testing that Ruby works. Let’s assume it does. That leaves the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Here’s the Star Trek Reference
This is the logic of the class, in terms of our spaceship metaphor, specifically a Starfleet ship: the Path ship receives a message from Starfleet to normalize the path. The
normalize_path method, the captain on the spaceship, calls on
relative_path?, the engineering officer on the spaceship. This method in turn calls on the science officer,
absolute_path?, who sends some report back to engineering, who interprets that message and in turn sends a more complete report back up to the captain. So far, all the action has been happening inside the spaceship, no transporters required. Furthermore, Starfleet doesn’t want to know about the contents of the reports from the engineering and science officers. We just want to know what the captain has decided. Finally,
normalize_path shoots the normalized path back out to Starfleet. The two tests we have written will pass.
The key point here is that if either the
absolute_path? method or the
relative_path? method was broken, we would receive an incorrect result in our test of
normalize_path. If engineering reports incorrect data, then the captain makes the wrong decision. If we can make the two tests that we have pass, we have effectively tested the two internal methods and assured that they work as intended. If we start to write tests for those two, it is a waste of time (although it is good practice for writing tests), because to produce separate tests for the helper methods we will end up recreating the scenario (creating one absolute path and one relative path and matching the outcomes to what we expect to happen). This process is also guilty of overspecification. Once we’ve written the method, we don’t really care how
relative_path? works, so if we start mandating that we must have a
false result, or any specific result at all, we have an artificial constraint. If we want to change the way those methods work in the future we will break these tests, meaning we have to write new tests and double our workload for that change. Our test suite turns into a burden that weighs us down rather than a safety net that frees us to make acrobatic leaps of Ruby.
I have neglected to mention so far any of the many caveats that Sandi applies to her process. Suffice it to say, if it makes more sense to break the rules, then it’s okay to break the rules. In a lot of cases, it is necessary to test complicated internal methods, to ensure that they are sound, but once we know they are working we can delete those tests. Aside from those and a few other exception cases, Sandi tells us that writing good tests is an exercise in minimalism. Most of the time we can do more with less.