Archive for the 'Hacking' Category

My experiences with TDD

Tuesday, April 20th, 2010

I recently wrapped up a project at my workplace. It was an RPC mechanism for a usb gadget running an embedded UNIX. The framework exposed the device as a software object on which one could make regular function calls and stuff. They would be translated into an application protocol, sent to the device, executed there and the results returned.

The framework was useful but not particularly novel. It was done mostly in Python with a few C extensions here and there to work around the limitations of the struct module. The interesting part was that it was the first real world non-trivial application that I did entirely using TDD (test driven development). I prepared myself with a copy of TDD by example by Kent Beck which was an excellent book. This blog post puts down in writing some of my feelings and experiences with the whole business.

  1. It’s time consuming
  2. The red/green/refactor cycle is addictive and after a while becomes second nature but it is time consuming. I easily took around 2 or 3 times the time to develop the app that I would have taken if I were just coding and doing manual testing. Of course, there is effort to write tests and it paid off substantially. Massive changes could be easily verified, git bisect became useful and my confidence in the code and willingness to refactor it increased.

  3. Design improvements and cleaner abstractions
  4. The need to build the app piece by testable piece resulted in an architecture that was very loosely coupled and relied on some simple API conventions. Also, while the actual device and host communicated via. USB, it would be tough to test using that. To manage this, I abstracted the communication channel from the actual application and made a dummy ’short circuit’ channel to do testing of the two endpoints. I don’t think I’d have done this if I didn’t have to test it so rigorously.

  5. Smaller/cleaner code
  6. Functions in the application were small and clean. The interfaces were clearly specified and didn’t keep any local state (since I needed to drive them from the tests). I didn’t spend that much time working on the cleanliness of the tests themselves so that part accumulated some cruft but the app itself was quite clean and is still very readable.

  7. Shared culture
  8. It’s important to have everyone in the team work the same way. I went off for 2 weeks during the project and got back only to find most of my abstractions broken and some 30 tests out of the 42 odd ones in the suite failing. This was depressing and was a huge mental barrier against further development. It totally shattered the confidence I had so carefully built up.

  9. Nice APIs
  10. Even before you write the app, you will write the tests and there you’ll be forced to write a nice API so that you can make the test readable. This forces you to design your APIs better than you would have otherwise. The main reason is that you have a use case for your APIs before you’ve designed them

  11. Brain in a jar testing is bad and almost useless
  12. If you mock out all dependencies a module or class has and test it in isolation, chances are that you’ll get flimsy tests that don’t really do anything useful. There should be a significant piece that you are testing. Too small might mean too trivial.

  13. Scope of tests
  14. If you’re testing a method, it’s best to by default test inputs, outputs and side effects. Anything less and your coverage is likely to suffer.

  15. Testing is under emphasised
  16. Testing, build harnesses, infrastructure etc. are as important as if not more so than the actual application being built. People need to get this into their heads.

  17. Don’t stop at unit tests
  18. 4 modules with around 60 tests were done. Every line was tested. The whole thing was put together and it crashed and burnt. Integration tests that verify end to end behaviour are a must and need to be done as part of the regular test suite. It might be hard to develop them before the app is done though. They don’t form a part of the whole TDD cycle but they’re necessary.

  19. It’s fun
  20. This alone should be a sufficient reason why I’m going to use this approach for my next project as well.

The app has been deployed and is in production. Let’s see how things go.

PyCon Atlanta 2010

Wednesday, March 10th, 2010

Python community service award

Friday, December 4th, 2009

A review of FOSSConf

Saturday, October 17th, 2009

Bootable Jaunty USB stick

Sunday, September 13th, 2009