Tool Upgrade

We upgraded almost all our development tools in this latest change cycle. This is nice for developers. We like to use recent versions of the toolsets. This is a nightmare for developers. When you change multiple tools at once, it is hard to identify which change was the cause for any given bug.

Here are the tools we changed during this pass. Moved up to the Oracle 11g client. Upgraded to Microsoft Visual Studio 2008. Jumped up to Installshield version 18. Also grabbed a recent copy of the Objective Toolkit. Ouch. Everything is changing.

The first pain was that the new Installshield was so much newer than our ancient copy, we had to write the install scripts from scratch. That was a quick weekend hack that just did not work. I spent a lot of time reverse engineering the pieces of the install that were missing from our rewrite.

Next we had some pain from Visual Studio. Installing the runtimes was not as simple as dropping some DLLs in our application directory. Turns out we either need to run the Microsoft installer, or use the support supplied from Installshield. Growing pains really hurt.

The third party tool upgrade required some work. There was some disturbia initially when the developers could not quickly figure out how to build the third party components. We had to do some due diligence to figure out how to build the darn thing. Once that was solved, we quickly figured out how to use the components and deploy the minimal set of third party runtimes.

I am just happy that I only spent about a month or so dealing with all the upgrade problems. Another developer took over and spent an additional month or two working through the problems.

Hard Coded

I got called to an emergency meeting this week. The configuration data in our production system was not correct. There were some missing values. That called all the data into question. There were a bunch of questions about some values of suspect nature.

I waited for a while. The guilty parties did not speak up. So I piped up that development did not have time to be thorough with some late development. I knew because I got called in at the last minute to help the developer figure out what the module was doing.

So the developer hard coded all values this time around. It appears he also left a remnant of the configuration data in the database. However this data was unused. Therefore the values in the config table did not matter. A manager was upset that he was not in the loop about these decisions.

Finally the developer who did the dirty work did pipe up. He said that all his decisions were peer reviewed by managers. Everything was written out and reviewed. I guess that really means the manager approved something he did not comprehend. Next time pay attention man.

I was tasked with putting together a plan to get us out of this mess. First step was to get the needed production configuration values out there and verified. Next we will need the developer to spend the time to use config parameters to drive the module. We will need a lot of such parameters. Then we will need to document all this stuff.

I like to be conservative. So I propsed we implement this is the next major release. And when we do roll this thing out, I will grab all the managers and explain the ramifications of everything we are doing. Luckily it is easier to explain that we are doing things the right way finally.

Information Hiding

We have a lot of stored procedures in our system. They are pretty much bunbled together in packages. Since our database is Oracle, the packages had a specification and a body. The spec is the view of exported procedures to the outside world. While the body contains the implementation along with internal helper functions.

This is a pretty good setup. You only see procedures you can use in the spec. Details are hidden away in the body. You would think this is some good design. Well most of our job consists of identifying and correcting bugs in the software. This is difficult when the bug is in a stored procedure housed in a package. Many bugs only show up in production. That version of the code is secured very tightly.

Often times we need to run the code and output some debug statements. Usually we will extract the portion of code that is in question into a stand alone script. This is easy if the code extracted does not reference other private functions in the body. But if it does, you got a lot more work to do during extraction.

I often find myself making a lot more visible in the spec than I immediately need. This is because I anticipate the need to make a call to some of the helper functions from the outside for testing or debugging purposes. However that defeats the whole purpose of information hiding in the first place. It is almost a catch 22.

Artificial Intelligence on the Sports Scene

There is a startup called Statsheet that is doing sports coverage for Division I basketball teams. The surprising thing is that the updates are all done by Artificial Intelligence. However the come out looking like they are written by a human being.

The project is around 6 months old. A program takes box score data and turns it into text that is looks just like a human wrote it. I tell you some samples I saw were amazing.

Great. Just when the reporting industry is about to go out of business, RoboReporter comes and put the nails on the coffin. There is a benefit. Even no name basketball teams are going to get great coverage. At least mom will be happy when she reads the report on Junior's basketball win.

Learning how to Code

How do you learn to be a good coder? Well you can get somebody else's code and hack it. That's how I started learning. You can also dive into someone else's unit tests to figure out what they are doing.

It is best to inspect a small project's code. That way you will not get overwhelmed or lost. Check out code from dudes who are gurus at software. You should also look at the source for any libs you use. Trace down what happens when you make API calls.

Reading other people's code is an easy way to learn what to do when you code yourself. However the very best way to learn coding is to just do it. There are many apps waiting to be written. What are you waiting for?

MultiCore Design

Virtualization let's you scale applications on multiple processors. This allows multiple nodes to run in parallel, giving you a performance gain. You might have 10 times as many instances of your application running. This is easy to set up, but might not be optimal speed wise. It works best if there are a small number of dependencies in your application.

The problem with this technique is that hardware is no longer getting exponential increases in power and capacity. Therefore you would be better off designing your applications for the multicore processors out there. Normally this means using a threading model which might be complicated. You can cheat a little by using a thread pool with a bunch of worker threads.

There is not usually a need to entirely rewrite your applications to make them support multicore. You got to put some synchronization. And you must make sure race conditions and deadlocks do not occur in your code. Model driven development can help with this task. Refactor your apps for multicore usage, and you will see a big gain in performance.

Software Engineering Method and Theory

There are a bunch of industry veteran trying to discern the universal truths of software development. They have formed the Software Engineering Method and Theory group. It is called SEMAT for short. Some industry luminaries like Ivan Jacobson, Bertrand Meyer, and Larry Constantine are involved.

This group wants to investigate the core theories and best practices in the development world. They state that this group is not about any specific product of standard. They are attempting to uncover the universal set of practices that occur in software development.

SEMAT has broken out six working groups. These include terminology, kernel language, universals, core theory, requirements, and assessment. The group reiterates that it is no looking to create a new development methodology. Instead they are defining software development patterns and practices.

There are some who have their doubts about this effort. Universities lag the industry in practice. It make take many years for the work of this group to bear fruit. This is a theoretical exercise. There may not be any impact to real world development. Something like this was tried before with the Agile Manifesto.