Missing an Opportunity

Last month, I gave a talk to a company about Selenium testing. The talk was about some of the basics of starting to use Selenium, and some of the lessons I’ve learned about using it (see my previous two entries about some of my opinions). However, I got a question during the Q&A session, “Why do I need to refactor the tests and create all of those helper methods when I can just use grep and change things as needed?”, which I did not do a good job of answering. I keep replaying my answer in my head and wish that I had said more.

My answer was basically, that it is a real pain to just use grep, and how do you know that you’re changing the correct instances of the code? Their response, which is most likely accurate, is that they know and are comfortable with grep, and it solves their need.

My regret was that I should have stressed that the reader and maintainer of the tests should be able to look at the tests and understand what it’s trying to do and validate. If you have a set of Selenium steps that are 20 or 30 (I’ve seen 50) lines long, how is the maintainer of that code supposed to know what’s going on? How do they know which time in that sequence of steps, especially if there is duplicate Selenium, they should edit?

If I were asked that question again, I would focus more on the readability and maintainability of the code rather than getting into semantics of the use of grep. I would also refer the questioner to Clean Code: A Handbook of Agile Software Craftsmanship by Rob Martin and his story about how a team he consulted for which treated test code as a subclass and didn’t maintain the code. Soon their tests were no longer useful and had to be discarded because they were not held to the same standards as their production code.

Another solution for this problem, which I did not cover in the talk (out of scope for this discussion), would be to introduce the page pattern from Selenium 2. Even though I have not used it in production, I have found the idea very intriguing and think it can be helpful to teams starting up their web UI’s automated testing. There are several frameworks that have sprouted up using this pattern, most notably using Spok and Geb together.

My Favorite Technical Books

Over the years, my family has given me a lot of grief for reading technical books on the beach, during the holidays, or pretty much anytime. I read a lot of these books, and find them to be helpful in my everyday job as a developer or manager.

The most important characteristics of a good technical book include:

·      Well Written. I’ll freely admit that I’m not the world’s best writer. However, there are some books that need a good editor. There are typos and sentences where after the fourth or fifth reading I still don’t understand. My mother was an editor and writer for technical manuals, I just wish more authors would team up with a good writer more frequently. One example of a really well written book is The Well-Grounded Rubyist. It is a bit out of date now (pretty much any Ruby book is out of date by the time it’s printed), but the way David Black is able to express what he’s teaching should be duplicated in other books.

·      Concise.  I felt that the message, and lessons, in Domain Driven Design: Tackling Complexity in the Heart of Software by Eric Evans, are extremely important, but the verboseness of the book made it difficult to find those lessons. I find that the books, which give examples of their points in a clean and concise manner to demonstrate their points, tend to resonate with me.

·      Reusable. Don’t get me wrong, I like reading about a specific technology and doing a deep dive with it (I have version 2 and 3 of Spring in Action by Craig Walls and like them a lot), but a truly transformative book should be able to be used regardless of the specific technology.

Of all of the books that I’ve read, I would place the following three atop of the list:

1.     Refactoring: Improving the Design of Existing Code by Martin Fowler. This was the first technical book that was not about a specific technology, but rather about a mindset. This book is the one that made me start reading more technology books, not so much for the specific technology, but rather for the patterns I can use to go across different technologies. Even to this day, I have this book on my desk at work and will often open it, or refer to it. It is by far the most dog-eared book in my collection.

2.     Practices of an Agile Developer by Venkat Subramaniam and Andy Hunt. This is a very easy read, but probably has more lessons and topics covered per page than any book I’ve ever read. It is not a deep dive on any one topic, but covers many topics in an easy to follow and easily reference able manner. I liked this book as a kick off for other topics like TDD, pair programming and many other topics that interest me. While I was a manager, I gave this book to each of my new employees during their first week as homework. Each person learned something new and helped frame our discussions about how we work. On a side note, I often got the comment, “hey we’re doing a lot of the things in this book”, to which I just replied, “yup”.

3.     Clean Code: A Handbook of Agile Software Craftsmanship by Rob Martin. I only just recently read this book, but I wish I had read it sooner (like my first job out of college sooner). This book covers a wide range of topics, but the number one I came away with is, take pride of your work. Write your code in a way that when you look back at it in six months later, you still understand what’s going on because your code speaks to you. I have a suspicion that this book will soon be as dog-eared as Refactoring. If you’re a developer (in any technology), pick up this book and read through it, become a craftsman of your technology, and make the lives easier on those who must maintain your code after you’ve moved on.

Some other of my favorite books: Succeeding with Agile: Software Development Using Scrum by Mike Cohn, Dreaming in Code by Scott Rosenberg, Continuous Delivery by Jez Humble and David Farley, Driving Technical Change by Terrence Ryan, The Paradox of Choice: Why More is Less by Barry Schwartz, Domain Driven Design: Tackling Complexity in the Heart of Software by Eric Evans, Art of Agile Development by James Shore and Chromatic, Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen, Design Patterns by the Gang of Four, and Career 2.0: Take Control of Your Life by Jared Richardson.

If you have some favorites, I would love to know what they are as I’m always looking for more to read. I just started Working Effectively with Legacy Code by Michael Feathers, and have Growing Object Oriented Software, Guided by Tests by Nat Pryce in my queue.

Selenium: What to do

In my last post, I went through some of the pros and cons of Selenium.

Now I'd like to go through some of the lessons I learned to keep Selenium tests maintainable and valuable for a team.

Run continuously. How often does a UX developer change the user experience? Isn't that their job? When things change in the UI, Selenium will break, that is the downside using the UI to drive our tests.  With CI servers becoming more and more popular, groups have gotten into the habit of running their tests after every commit, otherwise they rust. Why don't they do it with their Selenium tests? I have found after all of the other tests pass, kicking off your Selenium tests is the logical next step. We avoid test rust, a huge Herculean effort at the end of an iteration to make the tests all pass again when there is always so much time, and reduce the feedback cycle as much as possible.

Don't over reach. Do not make too many tests. These tests take a while to run. The more tests you create, the longer they take and the longer the feedback cycle. I've found that only testing your regression tests, or acceptance tests, is more than enough. By only testing the basics, you allow your QA members to do some exploratory testing to ensure that the application is working.

Practice the DRY principal religiously. Let me give you an example. You have an e-commerce site and the developers change something in step one of your checkout flow, but you have ten test which go through this flow. Do you want to find every place that you're calling the newly changed code? Trust me, hunting these references down becomes a wicked time sink. Instead, if you have a step which is repeated more than once, create a method for that step. You'll be happy that you do. which leads me to

Constantly refactor. These are not just some test scripts which people don't need to pay attention to. You will end up creating a DSL for your website with Selenium as the driver. If a method's name doesn't make sense, change it.. If there is code repetition, combine it. If there is any code which is not clear, refactor. You may not have your most senior people maintaining this code, but to reduce the costs, make sure they have opportunities to review this code.

Find ways to make the tests faster. Probably part of the last point, but it bears repeating. These tests are slow. If you can combine tests, do it. If you don't need to run ten times through a step, do it. If you can create data needed for the tests directly with a database connection, do it. Or if you can clan up after the tests without going through the UI, do it. The faster the tests, the shorter that feedback loop, and the smaller the costs are to your group to create, run and maintain these tests.

Use Web driver. With web driver, you don't need to start up an extra server in order to run your tests. Also, Selenium 2 uses web driver and it really is a big step forward.

If you have any question or comments, please don't hesitate to contact me.

Pros and Cons of Selenium

Over the last few projects I have worked on, there has been a need for testing the web UI. Doing manual testing on the UI is expensive and time consuming (never mind that many bugs are ignored as rerunning the same tests over and over again is boring). Many groups have started using Selenium to do the web UI testing with mixed results.

Selenium (http://seleniumhq.org/) is a great tool and does what it's designed to do very well. Selenium is very good for doing directed tests where there are not a lot of steps. Anything more complicated than that, and Selenium starts to give you troubles. Selenium is also very easy to get set up and going, especially with Selenium IDE and Webdriver. Another great point of Selenium is that you can use almost any popular programming language from Java to Ruby to Python.

However, there are some concerns of using Selenium as well. First, they are slow. Usually, Selenium tests need to start the browser and stop the browser between each test. Another factor which increases the length of the tests is that there are a lot of sleeps or pauses in the the test while waiting for an element to become present. This slowness will cause the tests to be run less and less frequently creating tests which rust and become expensive to maintain.

Second, they are brittle and non-deterministic. Often times the test will break because of a network failure, or a page took too long to load. When the test is run again, it will pass. This causes a lot of problems when you're trying to rely on the tests as there tends to be a lot of extra noise caused by this brittleness. How do you know if a test is actually broken, or the site had a hiccup?

Finally, Selenium tests are easy to create. Wait, What? You just said that was one of the good things of Selenium. Well, yes, ease of creating tests on it's own is a good thing. However, what happens when a developer goes and changes some piece of functionality on your site and suddenly, a whole slew of tests break? Also, the more tests you create, also means the longer the tests take to run (see my previous point) meaning it takes longer to close the feedback loop.

My next post will talk about some of the lessons I learned on how to overcome some of these issues.

Introducing "Slack"

Over the last few years of working in agile teams, I have struggled with reducing technical debt. Most of the projects I've been a part of, teams are so focused on “getting things done” that taking the time to focus on fixing technical debt was out of scope. As you can imagine, these projects started off fast, but quickly got bogged down by our technical debt. This lead to cutting more corners (increasing our debt even further) to maintain our velocity, or convincing our product owner to take an iteration or two to focus solely on reducing our debt. None of the techniques I have tried were successful: adding technical stories to the backlog were constantly ignored or pushed back as lower priority, midnight projects just caused me to become burned out, or just ignoring it simply made the problem worse.

I had begun to think this was an issue which was endemic to Scrum.

Last week, I participated in the Art of Agile Planning and the Art of Agile Development, expertly lead by James Shore (author of The Art of Agile Development) and Diana Larsen (author of Agile Retrospectives). This course is chock full of advice, ideas and lessons on agile projects (I can't recommend this course enough if you're lucky enough to have an opportunity).

During the sessions, James Shore repeatedly stressed that reducing technical debt was not a task to be done in huge blocks of time, requiring technical stories and after hour work, but rather an on going activity. Reducing technical debt should be done in the same manner as TDD and refactoring, small incremental steps (wow is this a common theme across agile). James took this philosophy one step further and recommended we also introduce “slack” (I don't like the term because of it's negative connotation or it's closeness to sand bagging, but I can't come up with a better term).

What James recommends is that you take your velocity and if it went down in the previous sprint, you cannot take on more points than what was accomplished previously. Also, for several sprints to follow, you cannot raise your velocity either. As your team is working on stories, they look for areas where they can reduce their technical debt. Only when the team agrees that they can increase their velocity after they feel their debt is manageable (if not paid in full). Teams will find that their velocity is higher than it had been before introducing “slack”, and will also continue to rise to areas which are higher then they ever previously attained. This entire cycle may take a few months, but as in paying your credit card debt, will pay for itself.

What do you think of this idea? Would you be willing to try this on your team? Have you tried it on your team? If so, what were your results?

Career 2.0 Video

Preparing to present Jared Richardson's video on Career 2.0. http://qik.com/video/1009098. Thanks Jared for making this public. If you don't like to watch videos, read his book: http://career20.blogspot.com/

I think this is a great video that everyone who is a developer should watch to make sure they still have a job in 5, 10, 15 years from now.

JUnit and Parameterized tests

So, over the past few weeks, I have been playing with JUnit 4's Parameterized tests, and found them to be extremely useful in cases that you want to throw a lot of different values at a method to make sure you've covered all of your bases. A great, simple, example of this is when you want to test an equals method in your code.

Let's take a class Person which looks like this:

public class Person {
    private final String firstName;
    private final String lastName;
    private final long id;
    public Person (String firstName, String lastName, long id) {
        this.firstName = firstName;
        this.lastName = lastName;
        this.id = id;
    }
    ... 
    (getters, hashCode, toString go here)

    public boolean equals(Object obj) {
       if (this == obj)
           return true;
       if (obj == null)
           return false;
       if (getClass() != obj.getClass())
           return false;
       Person other = (Person) obj;
       if (firstName == null) {
           if (other.firstName != null)
               return false;
       } else if (!firstName.equals(other.firstName))
           return false;
       if (lastName == null) {
           if (other.lastName != null)
               return false;
       } else if (!lastName.equals(other.lastName))
           return false;
       if (id != other.id) 
           return false;
       return true;
   }
}

Now, you can create a test, or series of tests in a method to do this, but that is cumbersome. I found this an easy way to miss tests, so I looked to something different. Parameterized tests were actually really easy to set up and get going, so I created something like this:

@RunWith(Parameterized.class)
public class PersonEqualsTest {

    @SuppressWarnings("unchecked")
    @Parameters
    public static Collection data() {
        //
        return Arrays.asList(new Object[][] {
                {
                    new Person("Jane", "doe", 1234), null, false 
                },
                {
                    new Person("Jane", "doe", 1234),
                    new Person("Jane", "doe", 1234), 
                    true 
                },
                {
                    new Person("Jane", "doe", 1234),
                    new Person("Jack", "doe", 1234), 
                    false 
                },
... (more variations of the above going through each case)
        });
    }

    private final Person expected;
    private final Person actual;
    private final boolean expectedResult;

    public PersonEqualsTest(Person expected, Person actual, boolean expectedResult) {
        this.expected = expected;
        this.actual = actual;
        this.expectedResult = expectedResult;
    }

    @Test
    public void testEqualsObject() {
        assertEquals("expected: " + expected.toString() + " but was " + actual, expectedResult, expected.equals(actual));
    }

}

If I need to add a new way to test the equals method, I can just add a new array object to the data method. Very quick and easy, and makes testing a method with a high cyclomatic complexity, much more simple. This works really well when using Mock objects with JMock or EasyMock and testing multiple results.

There are a few limitations to this approach. First, you can only test one method at a time. If you try doing multiple tests, you will run each test as many times as you have objects in your data array. Second, the results can be hard to read. It is very important to have good error messages in your assertions, otherwise tracking down which test broke can be difficult. Finally, it is easy over use these tests. Like all new technologies, it is easy to try and use this test case for all tests (this is a bad idea). Find out what makes sense for you and go from there.

Please let me know if you have any questions, or clarifications needed for this post, as I will be happy to help.

Jon