Missing an Opportunity

Last month, I gave a talk to a company about Selenium testing. The talk was about some of the basics of starting to use Selenium, and some of the lessons I’ve learned about using it (see my previous two entries about some of my opinions). However, I got a question during the Q&A session, “Why do I need to refactor the tests and create all of those helper methods when I can just use grep and change things as needed?”, which I did not do a good job of answering. I keep replaying my answer in my head and wish that I had said more.

My answer was basically, that it is a real pain to just use grep, and how do you know that you’re changing the correct instances of the code? Their response, which is most likely accurate, is that they know and are comfortable with grep, and it solves their need.

My regret was that I should have stressed that the reader and maintainer of the tests should be able to look at the tests and understand what it’s trying to do and validate. If you have a set of Selenium steps that are 20 or 30 (I’ve seen 50) lines long, how is the maintainer of that code supposed to know what’s going on? How do they know which time in that sequence of steps, especially if there is duplicate Selenium, they should edit?

If I were asked that question again, I would focus more on the readability and maintainability of the code rather than getting into semantics of the use of grep. I would also refer the questioner to Clean Code: A Handbook of Agile Software Craftsmanship by Rob Martin and his story about how a team he consulted for which treated test code as a subclass and didn’t maintain the code. Soon their tests were no longer useful and had to be discarded because they were not held to the same standards as their production code.

Another solution for this problem, which I did not cover in the talk (out of scope for this discussion), would be to introduce the page pattern from Selenium 2. Even though I have not used it in production, I have found the idea very intriguing and think it can be helpful to teams starting up their web UI’s automated testing. There are several frameworks that have sprouted up using this pattern, most notably using Spok and Geb together.