Journal Zarf's Journal: Losing Perl (part 2) 10

Last time, I mentioned that internal office politics were driving the adoption of a different programming language at work. We've been a Perl shop for the last five years and the three primary programmers at my shop have 10, 8, and 5 years programming experience in Perl. All this experience is working against us and hurting our end product.

One of our developers has never programmed in any other language or in any other environment. He's spent the last five years with this company and has enjoyed an honored place in the company. He learned to program for the first time in 2000 while working for this company and he is responsible for 75% of the legacy code on the system. He's very good with the current code base and can solve problems in it very rapidly.

I was brought in about a year ago to do development and design work for these legacy systems. It became apparent that there were several horrible design flaws in the legacy system that jeopardized data integrity and created a great deal of confusion amongst users and management.

The site's back end UI was also very arcane. It was traditional CGI and frequently mixed HTML inside Perl. The result were hundreds (yes hundreds) of nearly impossible to maintain pages with logic and display spread between HTML templates, Perl, and Javascript often on the same screen. To change a single element's behavior required altering that element in the Database, Javascript, HTML, and Perl.

Our current staff has become so incredibly efficient at doing this work in triplicate it is second nature to continuing making these same choices, over and over, propagating the same errors in design. It is much faster to produce a poorly working product using the existing system than it is to engineer a "good" solution.

I spent the last year tracing the Data Model and finding that it was not even in first normal form... which is a real feat but entirely possible... as I normalized one entity I would have to follow it through all the HTML, Javascript, and Perl that touched it. A single column change could take weeks to fully account for. Many times testing would fail to include effects of the passage of time or extreme load.

Users have become so used to the constant instability of the system that they no longer complain about usability. One section of the Information System is notorious for taking data in, but never returning it properly. The data enters the Database and is transformed, it affects other screens, but somehow corrupts itself.

The password management system will chaotically remember and forget passwords. It sometimes forgets that a password was changed, sometimes it applies the change a random number of days later.

All these problems are not caused by failures in the off the shelf software, the server hardware, or failing disk drives. Amazingly, all these intermittent and buggy behaviors are directly related to data integrity issues due to poor software and database design. It is the lack of design itself that causes one person to report sixty-seven children when there should only be one.

Our current staff has learned to be lazy with Perl. Lazy in a bad way not the Larry Wall way. The current culture of laziness doesn't save a future self from work, it is the selfish laziness that seeks to finish a project today no matter what the solution may do to damage the overall system.

When you read a requirements document and decide that there could only ever be a need for three plane tickets per person and build your tables to hold only three plane tickets... that's lazy and selfish. When you get a request for a fourth plane ticket position and add a fourth plane ticket column to the widening table, that's lazy and stupid. Creating a tickets table that relates a person to any number of tickets is lazy and smart. The more flexible design means that you never have to revisit the application again to accommodate a fifth ticket.

The best possible thing to do for the existing system is to either terminate it or the developers who built it. Starting over again could only lead to a better system. If we wish to retain our current staff, then something drastic must happen to destroy their old method of thinking. The single most drastic thing that could happen is to remove all their familiar tools, force them to retrain, and learn a new way of doing software.

We will end up losing Perl in favor of Java. The freedom of Perl has been abused and now we will restrict the expressiveness of our programmers by imposing Java and the Java way on them. By dumbing down our development environment we can control how much positive or negative effect a single programmer can have. There will only one right way to do anything in the new environment.

You trade freedom for stability.