Maintenance - Repay the debt

It's time to repay some of that technical debt you have accumulated. The project has been running for a few years and often the code gets smelly. The code smells of complexity and bugs.

What do you do to approach such a problem? You clean it up of course!

Now I will tell you how you can choose what areas in the code you want to clean. For this you need a couple of tools. First one to calculate complexity and then you need something that tells you about changes in your codebase.

I have used Complexian and StatSVN in this example, but you may find other tools that does the same job just as well.

I started with Complexian and found that a few classes had a complexity above 21, which I consider as high risk. Then I combined those data with what we can get from StatSVN and came up with more that suggested that we should focus on the classes that has high complexity and many revisions. This because I reckon that a class with high complexity that is changed a lot must mean trouble. There is a lot of different approaches you can use, but this one proved to be sufficiant in my case.

From Complexian
Register.java: Complexity is 38

From StatSVN
Register.java : Lines of code: 2273
Register.java : Revisions: 32 (4th place)


Here you can of course add all the static code analysis you want, but for me this is sufficient to tell that there is something wrong with this class. In this way it is also easy to persuade the project manager that this is a risk that needs to be taken care of.

Then I dive in to the class to find if's and else's that needs a refactoring.. At the same time I notice that the class is in the wrong package as well. My goal is to reduce the risk in this class. Never refactor without a goal. Refactoring makes no sense if you do not do it to achieve something better. If the class never gets changed and you have no bugs that you know of in there, leave it alone. It would probably not show up as a candidate either since there would not be many revisions of that file.

Write some tests for those methods that have high complexity is hard, but rewarding. In this way you deal with risk in a professional manner instead of just "feeling" where to do you cleaning.

Hope this gives someone inspiration and a way to start repaying that technical debt they have accumulated during the years.

The Importance of Removing Impediments

What is an impediment?

I do not actually know, because it is such a difficult word. But what I do know is that the time I have to wait for a build to finish is not productive for me. It even costs the customer money. This is for me an impediment.

If the time it takes to build the application is too long it starts imposing challenges on me. When it takes too long I do not want to build too often.. If i do not build too often, the tests is not run often and I will not commit code very often. This leads to finding bugs much later than I normally would have if I had ran the build more often. So I use more time fixing the bug because I have produced much more code since i made the bug somewhere in the huge codebase.

Did you get it? If not let's run som numbers on it instead. Lets say that it takes 8 minutes to build and that you use most of the resources on your computer while doing it. The best you do is drink a cup of coffee or read up on some documentation while you wait. Let's say you do the cup of coffee and wait for 8 minutes to see if the tests fail or pass. Its not only you in the project, but ten people that does the same thing. We all do it 2 times a day, because it takes so much time and we dont like coffee that much. We do it every day of course. So the numbers become:

8 minutes x 2 times x 10 people x 5 days = 800 minutes = 13.3 hours

Not only does it take you 16 minutes a day to build, but it takes the whole project 13.3 hours to build a week. Let us then say that you work 40 weeks a year and your project runs for 2 years.

13.3 hours x 80 weeks = 1064 hours

So during the lifespan of your project it has been used 1064 hours for running the build. What if those hours could have been used to implement more functionality because you have so much good code that your build did not take any noticable time at all. Imagine that the cost where 1000 NOK per hour that would amount to 1064000 NOK used on build/coffee during the project.

1064 hours x 1000 NOK = 1064000 NOK

That is not just an impediment to me, but to the customer. Build time matters! So remove those impediments while you still have time left to do so. If you are not busy building the application of course...

Visibility

I have just ordered a 32" flatscreen for our project! I plan to use it to increase the visibility of the continous integration status. By this I mean the build status, the Cobertura test coverage data and the JUnit report.

This means that when we have the daily meeting, the screen will be very visible and as scrum master I want to increase focus on our test coverage. More coverage will enable us to make changes without the fear of breaking the application.

The last week I have used several hours to remove broken and useless tests from the codebase. This has been tedious work and the result is a lower test coverage than earlier, but now we have only tests that are running and not giving us an error (not the same as failure). We have made tests during our workshops and these tests now shows brightly on the JUnit report (-sky) as passed tests. Hopefully this will be motivation to actually go in to old code and bring it up to better standards with tests. We do all these changes one small step at the time and it actually seem to be working!


I will be back with some pictures when the screen has arrived.

Workshop: Legacy Code

It's true. We got it! We got legacy code..

To best describe legacy code I turn to one of the guru's on the subject, Michael Feathers:

[After all, there is an emotionally neutral definition of "legacy code." Legacy code is code from the past, maintained because it works. But, for people who deal with it day in and day out "legacy code" is a pandora's box: sleepless nights and anxious days poring through bad structure, code that works in some incomprehensible way, days adding features with no way of estimating how long it will take. The age of the code has nothing to do with it. People are writing legacy code right now, maybe on your project.

The main thing that distinguishes legacy code from non-legacy code is tests, or rather a lack of tests.] ( full article and book )

We now have more than 180 000 lines of java code in my project. To me that is a lot of code. How can you have knowledge of the code to safely say that your change will not break any existing functionality? You can't!

There are many reasons to why you can't know for sure. Let me give you an example. One of the classes is almost 10 000 lines long and has methods that contains more than 500 lines of code. If we get a bug report and go in to that class to change something you can be sure that there is something in there waiting for you to mess it up. The class is impossible to change and it will take forever to fix this bug. And by fixing the bug you may actually introduce another , because someone was expecting that specific value in another part of the application.

The reason for expecting the bugged value can be many, but you can't know for sure if it will break something. Without a test, we can never know if that other functionality break when we change something.

Even with tests you can't be sure not to break anything. But at least your tests will lower the risk of not finding the bug you created with your change.


The tests help us have actual knowledge of the application. Not only can we read the tests, but we can actually get feedback from them when we do something to the code. This helps us to know more of the dependencies and the relations in the code. It helps us to understand the code better. But when you have 180 000 lines of code, do you really want to add another 100 000 in tests? No. But I do want to have tests around the changes i make and take small steps to create a safer place to work. Because it is fear that keeps us from attacking this monster class with our refactoring and our changes. Get rid of your fear by learning more about the code with tests.

In this spirit we have scheduled a small workshop for tomorrow to handle that fear. We will delete unused code and we will write tests for the code we are moving out of one of our biggest classes. We will discuss and learn about our code. With this knowledge of how to handle legacy code I hope that we will conquer our fear of changing code in horrible places.

This is knowledge in action!

TDD momentum continues

Just a small update this time. The TDD course seemed to go ok and many of the participants now have an understanding of how to actually write the tests first and then move on to the production code.

I got good feedback on the theory of the course, but the practical part was a bit unorganised and some found it difficult to think as proposed in TDD. This was great! It shows that we are truly not used to think this way and hopefully it will inspire to more.

I will refine my course to be even more streamlined when it comes to the practical part and hopefully with other employees we will not have the issue of an unfamiliar IDE.

For the .Net guys out there I came over this CruiseControl.Net demo today. So no excuses for not having continuous integration on your platform either!

I'll be back with a longer post on management issues when faced with delays which is out of your control. (promise!)

Test-Driven Development Momentum

Tomorrow I have, what we in Computas call, a competence-block with the new employees. These blocks contain information that we think that most employees in the company should know something about. Mine is about Test-Driven Development (TDD).

It all started last week when I was reading up on various blogs. Then I read "Learning The Holy Ways of Consulting" from Thomas F Nicholaisen and it spurred the idea of doing a TDD course for my two new project members. I started designing a course much like the one Thomas had and when I started talking about it at work people wanted to have such a course themselves as well.

Even though I have tried to implement this in my project before and not having much success (see: this post), I wanted to try again with the momentum of two new people. What I have now is an even better possibility, to show all new employees TDD and it's benefits. I mean it is extremely important that a company which not only produces code, but also maintains code has to use all quality-increasing tools available. Since we often maintain code for several years after the initial release anything else would be shooting yourself in the foot! For a good post as well as an example of old code renewal through TDD see Anders Norås' blog and his Ending a Legacy post.

Ok, let me get back on the actual subject of this post. The course will be held in two sessions. These two days being Monday and Tuesday from 9 to 11:30. During Monday I will go through some of the theory of Scrum, TDD and show them how to make a sprint log of the items I've made in Jira (thanks for the tip Thomas!). From there on, it's all practical. The task at hand will be the famous bowling assignment from uncle Bob as I'm using much of his ideas on TDD in the theory part of the course.

Lets hope that I will win them over to TDD and by doing this start something wonderful in our company. Maybe this is enough to create a critical mass of people to win over the rest? We will also of course make this competence-block available to all employees when I'm done with this prototype.

I'll tell you more later!