Python 2.6 final released

Whoot!

Python 2.6 final is now available.

12 Unit Testing Tips for Software Engineers

Unit Testing is one of the pillars of Agile Software Development. First introduced by Kent Beck, unit testing has found its way into the hearts and systems of many organizations. Unit tests help engineers reduce the number of bugs, hours spent on debugging, and contribute to healthier, more stable software.

In this post we look at a dozen unit testing tips that software engineers can apply, regardless of their programming language or environment.

1. Unit Test to Manage Your Risk

A newbie might ask Why should I write tests? Indeed, aren't tests boring stuff that software engineers want to outsource to those QA guys? That's a mentality that no longer has a place in modern software engineering. The goal of software teams is to produce software of the highest quality. Consumers and business users were rightly intolerant of buggy software of the 80s and 90s. But with the abundance of libraries, web services and integrated development environments that support refactoring and unit testing, there's now no excuse for software with bugs.

The idea behind unit testing is to create a set of tests for each software component. Unit tests facilitate continuous software testing; unlike manual tests, it's cheap to perform them repeatedly.

As your system expands, so does the body of unit tests. Each test is an insurance that the system works. Having a bug in the code means carrying a risk. Utilising a set of unit tests, engineers can dramatically reduce number of bugs and the risk with untested code.

2. Write a Test Case Per Major Component

When you start unit testing, always ask What Tests Should I Write?

The initial impulse is to write a bunch of functional tests; i.e., tests that probe different functions of the system. This is not correct. The right thing is to create a test case (a set of tests) for each major component.

The focus of the test is one component at a time. Within each component, look for an interface - a set of publicly exposed behaviour that component offers. You then should write at least one test per public method.

3. Create Abstract Test Case and Test Utilities

As with any code, there will be common things all your tests need to do. Start with finding a unit testing for your language. For example, in Java, engineers use JUnit - a simple yet powerful framework for writing tests in Java. The framework comes with TestCase class, the base class for all tests. Add convenient methods and utilities applicable to your environment. This way, all your tests cases can share this common infrastructure.

4. Write Smart Tests

Testing is time-consuming, so ensure your tests are effective. Good tests probe the core behaviour of each component, but do it with the least code possible. For example, there is very little reason in writing tests for Java Bean setter and getter methods, for these will be tested anyway.

Instead, write a test that focuses on the behaviour of the system. You don't need to be comprehensive; create the tests that come to mind now, then be ready to come back to add more.

5. Set up Clean Environment for Each Test

Software engineers are always concerned with efficiency, so when they hear that each test needs to be set up separately they worry about performance. Yet setting up each test correctly and from scratch is important. The last thing you want is for the test to fail because it used some old piece of data from another test. Ensure each test is set up properly and don't worry about efficiency.

In cases when you have a common environment for all tests - which doesn't change as tests run - you can add a static set up block to your base test class.

6. Use Mock Objects To Test Effectively

Setting up tests is not that simple; and at first glance sometimes seems impossible. For example, if using Amazon Web Services in your code, how can you simulate it in the test without impacting the real system?

There are a couple of ways. You can create fake data and use that in tests. In the system that has users, a special set of accounts can be utilised exclusively for testing.

Running tests against a production system is risky: what if something goes wrong and you delete actual user data? An alternative is fake data, called stubs or mock objects.

A mock object implements a particular interface, but returns predetermined results. For example, you can create a mock object for Amazon S3 which always reads files from your local disk. Mock objects are helpful when testing complex systems with lots of components. In Java, several frameworks help create mock objects, most notably JMock.

7. Refactor Tests When You Refactor the Code

Testing only pays if you really invest in it. Not only do you need to write tests, you also need to ensure they're up to date. When adding a new method to a component, you need to add one or more corresponding tests. Just like you should clean out unused code, also remove tests that are no longer applicable.

Unit tests are particularly helpful when doing large refactorings. Refactoring focuses on continuous sculpting of the code to help it stay correct. After you move code around and fix the tests, rerunning all the related tests ensures you didn't break anything while changing the system.

8. Write Tests Before Fixing a Bug

Unit tests are effective weapons in the fight against bugs. When you uncover a problem in your code, write a test that exposes this problem before fixing the code. This way, if the problem reappears, it will be caught with the test.

It is important to do this since you can't always write comprehensive tests right away. When you add a test for a bug, you're filling in the gap in your original tests in a disciplined way.

9. Use Unit Tests to Ensure Performance

In addition to guarding correctness of the code, unit tests can help ensure the performance of your code doesn't degrade over time. In many systems slowness creeps in as the system grows.

To write performance tests, you need to implement start and stop functions in your base test class. When appropriate you can use a time-particular method or code and assert that the elapsed time is within the limits of the desired performance.

10. Create Tests for Concurrent Code

Concurrent code is notoriously tricky and typically a source of many bugs. This is why it's important to unit test concurrent code. The way to do this is by using a system of sleeps and locks. You can write in sleep calls in your tests if you need to wait for a particular system state. While this is not a 100% correct solution, in many cases it's sufficient. To simulate concurrency in a more sophisticated scenario, you need to pass locks around to the objects you're testing. In doing so, you will be able to simulate concurrent system, but sequentially.

11. Run Tests Continuously

The whole point of tests is to run them a lot. Particularly in larger teams where dozens of developers are working on a common code base, continuous unit testing is important. You can set up tests to run every few hours or you can run them on each check-in of the code or just once a day (typically overnight). Decide which method is the most appropriate for your project and make the tests run automatically and continuously.

12. Have Fun Testing!

Probably the most important tip is to have fun. When I first encountered unit testing, I was sceptical and thought it was just extra work. But I gave it a chance, because smart people who I trusted told me that it's very useful.

Unit testing puts your brain into a state which is very different from coding state. It is challenging to think about what is a simple and correct set of tests for this given component.

Once you start writing tests, you'd wonder how you ever got by without them. To make tests even more fun, you can incorporate pair programming. Whether you get together with fellow engineers to write tests or write tests for each other's code, fun is guaranteed. At the end of the day, you will be comfortable knowing your system really works because your tests pass.

And now please join the conversation! Share unit testing lessons from your projects with all of us.

[From 12 Unit Testing Tips for Software Engineers]

Analog Rails - Thoughts from a Cad Manager

Last week myself and a senior member of my staff were invited to a hands-on look of Analog Rails. Analog Rails is company whose focus is to "create the best analog and RF environment for the IC circuit designer". The system is built upon Open Access and gives an analog design engineer the complete ability to do both schematic capture, simulation and layout. It nicely automates the more tedious aspects of analog layout while giving the design engineer the complete flexibilty. For example the process of both matching (common-centroid anyone) and wire-widening (based on via size) is done in very automated push button method.

In the last several months Analog Rails have really stepped up the development by starting to integrate with outside tools. For example they now integrated in Veritools into the design framework. This is a big step because it implies that they designed a flexible subsystem which is critical - there is no single solution in IC Design. I would like to see spectre ( -turbo of course ) and calibre integrated but it's not there yet.

Throughout the discussion / demo I began to understand the impact of this. Of course it's built on OA and will nicely integrate with Cadence and many many others, but something more interesting is afoot. This tool represents a clear methodology shift.. ..The demise of the block level analog layout engineer. Why? First as I said earlier, the layout automation piece is very intuitive and friendly. Plus the smaller technology geometries that analog design is pushing into (<90nm) is forcing simulation earlier (using Cadence) to account for device parasitics (LOD). So the analog engineer is already doing a fair amount of the placement and letting the layout guy clean up the work. But this tool is correct by construction and so the layout is clean from the get-go. So involving a layout person to "clean it up" isn't necessary. There will still need to be layout for macro / chip level integration but block level layout could (will) be going the way of hand-LVS.

There are areas which need to be better defined. From a cad managers perspective - I look at flexibility and integration. The flexibility to align the tool to corporate goals and strategy, and the hooks necessary to integrate it into my existing flow, and (perhaps) migration to this tool.

The flexibility of the tool is fundamentally there but needs better definition. For example: If the company has it's own fab, and the strategy is to use its own fab, how will this tool which claims to not need a PDK get the design constraints from my foundry into their system? The converse is also true - I am a fabless company with a strategy to tape-out to the lowest cost foundry - how do I ensure that the constraints from Dongbu will work with my design. The point is while Analog Rails clearly has the framework so support this, they need to have a clear methodology for how to implement design constraints which the customer may use or opt to use the defaults from Analog Rails.

[UPDATE]: I did receive a bunch of documentation on the methodology for creating these techfiles. It is pretty straight forward and not as difficult as I had imagined.

From the integration side it was better defined but still lacks the clarity for migration. Because the tool can work on OA it should "just work". But how does this system work when an existing layout done in Virtuoso or Laker which doesn't or isn't correct by construction adapt itself? We spoke of the "dummy mode" but I think a bit more clarity on this would be helpful. I not sure this is show stopper - because it would depend on the use model. If you define your flow/methodology such that this is the sole block level tool for a project it could work. If you want to do a mix - well that would have to be tested.

Overall I was very impressed. The tool has certainly matured over the last year. It's flexible sub-system and the ability to tie into other tools gives me reason to want to look further at the tool. It will be a real test to see this tool integrate with the work horses of the IC Design tools, and how easy it is to define a methodology around the tool. Again this tool has some really cool technology under the hood and should make for a fun evaluation. My concern (as any cad manager has) is that our designers are going to eat this up and really want it. That's a good problem to have.

iPhone Platform: What We Can Learn From Tap Tap Revenge

In reading this - it makes you consider what happens when having a marketing group becomes less important.... Get your fanatical customers to market your product for you and reward them by giving it to them for free.

...

But I think there is something even more important to notice about Tap Tap's success. When Seth Godin released his first marketing book, Permission Marketing, he gave away the first four chapters for free via pdf to over 150,000 people. It generated a lot of buzz about the book and was a big factor in the book's success when it eventually became available in hardback. So when he followed up with Unleashing The Ideavirus, he went one step further. He gave away the entire book for free in pdf (it still is available free). Seth says that over 2mm copies were given away. And yet when the book was published in hardback it went to #5 on Amazon. It was a demonstration of the very tactics Seth was evangelizing in the Ideavirus book. Seth asserted that by giving away your product early to your greatest fans who will do more work and put up with a less than optimal experience, you prime the pump for the mass market. That's because your early fans will spread the ideavirus and market the book for you.

Tapulous did the same thing with Tap Tap Revenge. The game was first made available for the iPhone at the start of this year. But the only people who could play it were people with phones that were jailbroken. So it's audience was small, but fanatical. These were the early iPhone app adopters, the ones who would work harder and put up with a less than optimal experience. But now that everyone can play Tap Tap, the early adopters are telling everyone else how great it is.

[From iPhone Platform: What We Can Learn From Tap Tap Revenge]

Thinking Like a Cocoa Programmer

Reading this should be the mantra of ANY software engineer.

Theocacao: “First and most importantly, the Cocoa programmer’s focus is always the end result for the user, not the academic sophistication of the code.”

[From Thinking Like a Cocoa Programmer]