Bulding Robust Systems

There are many principles you can employ to design systems that don't break often. This will help you sleep better at night. That is because the system won't crash, and they won't have to call you for help. I should know about this. I just got a support call this evening. We must not be following the right principles at work.

One this you can do is to manage your events with a queue. There is standard technology to deal with queues. An example is Microsoft's MSMQ. You can even choose a queue in the cloud if you want to get fancy. Wait. Hold that thought. You do not want the support calls. So choose a mature and stable queuing technology.

Many business processes are batch oriented. Each stage requires the output from the prior stage. Our back end is a big UNIX system. You would think you could just pipes the processes along. That works in theory. This is a bad practice in reality. You should design a scheduling system that spawns the processes separately. The results from each stage should go to a file. The scheduler can wake up every so often, see if a stage has completed, then start up the next stage. We have our loading software follow this pattern. It is rock solid, even when things go awry.

A final tip is one that I have heard before. Every time you resolve some trouble ticket, you should turn that into an additional automated unit test. You already are investing the time to test your bug fixes. It is a marginal extra effort to automate that testing. This helps you grow your unit test code base and coverage. Sometimes prevention helps you avoid the pain that you would get otherwise.

Scrum and DSDM

A magazine article I read recently reviewed different agile software methodologies. I have previously written about Extreme Programming. Now I will go over Scrum, and a new one I am not familiar with (DSDM).

Scrum is a management technique. One of its goals is to stabilize requirements. That alone is a noble goal. I find our requirements lacking in general. However Scrum also advocates a daily stand up meeting. Those I have found degenerating into a waste of a whole lot of time.

DSDM stands for Dynamic Systems Development Method. Have you ever heard of it? Neither have I until now. It attempts to hold the resources and delivery date constant. The way to achieve that is to modify the scope of the deliverable. I like the sound of that. It comes from Britain. I don't think it is popular in the States.

I will have more later on things like Continuous Integration and Test Driven Development.

Agile Methodology

I got a really good overview about Agile Methods from a magazine I read this weekend. Surprisingly it was a magazine on testing. It seems the techniques have progreesed past plain development to the world of software test.

Let's start with the first, and what I consider the biggest. That's XP which stands for Extreme Programming. It sounds funny when applied to testing. XP was created by a guy working for Chrysler.

XP focuses on delivering small release quickly. Then you can circle back and add more functionality in later releases. That has the distinct advantage of getting something done fast. It also provides a tangible program for the user to try out. You can get some great feedback from such a responsive development cycle.

There are a number of techniques used in the XP world. These include Test Driven Development (TDD), Pair Programming, and Refactoring. Perhaps I will go into these techniques at a later time. In the mean time, I plan on explaining Scrum and a new method called DSDM. Later.