Monday, November 4, 2013

A Short History of Free-Form RPG

Free-form RPG is back in the news with the announcement of free-form D, F, P, and H-specs. In this short essay, I look back at the history of free-form syntax in RPG.

It all started more than twenty years ago, back when RPG IV was being designed. At the time, there were two distinct schools of thought. On one side, there were people who insisted RPG should have a fully free-form syntax. On the other, were those who strongly believed that RPG should remain in its traditional fixed-form layout. The result was a compromise that leaned heavily towards a traditional fixed-form syntax, but with some free-form elements, such as keywords on the new definition specification and expressions in an extended factor two entry.

My first task in the implementation of the RPG IV compiler was coding the new D-spec, along with its keywords. Later, we were able to convince our planner that keywords should also be allowed on the F-spec. Later still, keywords on the H-spec became an obvious design change.

The extended factor two opcodes, such as EVAL, IF, DOW, etc., were a riskier proposition. At first one developer was assigned the task, and later another to help out the first. But as time went on, it was clear that they weren't making any progress. Late in the development cycle, I was assigned that piece of the language. Scrapping the work already done and starting from scratch, I finished the feature well in time for the initial release of RPG IV.

With the release of RPG IV, the development team was scaled back, leaving two developers working on new features. Because the extended factor two calcs were such a hit, one proposed enhancement was a fully free-form syntax for C-specs. However, we believed that it would require 100% of all development and testing resources for one release. When considering potential enhancements, this item would always get pushed well down the priority list.

This changed during the planning cycle for V5R1. I realized that common coding standards had changed somewhat during the past few years. I realized that a new free-form C-spec did not need to include all traditional C-spec features. Conditioning indicators weren't needed now that we had the IF opcode. Resulting indicators were largely deprecated in favor of built-in functions. And some opcodes were no longer needed, such as IFxx and the MOVE opcodes. We just needed a few additional built-in functions to fill in some gaps. With this strategy, free-form calcs could be implemented relatively easily, along with other useful functional enhancements.

But of course, this proposal was not without controversy. In order to get the proper design, we had approval from management to discuss the proposal publicly, on a popular RPG related mailing list. Many people loved the idea, but some were vehemently against the proposal. Personally, although I knew we had something good, I was so disheartened with the criticism that I gave the feature a 50/50 chance of making it to release. However, in 2001 it was released, and over time, most critics came around to accept the syntax.

With V5R1 and /FREE released, we started thinking about moving towards a free-form syntax for other spec types. However, while free-form syntax made a lot of sense for calc specs, we could see little benefit for the rest of the language. Other enhancements of a more functional nature were always considered more important.

In the summer of 2003, the iSeries group in the Toronto Lab could not escape the "staffing actions" rampant throughout the Software Group, and the RPG compiler development team was reduced to one person. I was moved to a new team responsible for PL/X, a compiler used internally within IBM for the development of mainframe software.

So now we come to the Fall of 2013, 12 years after the release of  free-form calcs, 18 years after the release of RPG IV. A few weeks ago came the announcement of free-form D, F, P, and H-specs in RPG. Although I'm happy to see this come about, I'm also puzzled. It seems all too anti-climactic, too little too late to save an anachronism of a programming language. In a previous missive entitled Is RPG Dead? The Autopsy, I list half a dozen features common to modern programming languages but missing from RPG. Note that free-form syntax is not included in that list. At the time I wrote that, significant parts of the language were still fixed-form, but I didn't consider the lack of a fully free syntax significant.

So what's the big deal about the additional free-form syntax? More than ten years ago when I was part of the RPG development team, we always looked for useful functional features to add to the language. That is, features that would help improve programmer productivity, or allow programmers to do things they couldn't do easily before. But that seemed to change about ten years ago. In the last release that I had any involvement with, one feature added was XML operations within the language. It wasn't as if programmers couldn't handle XML before since there were already XML API's in use. But as far as I could tell, the only reason the XML opcodes were added was because COBOL was getting XML operations, and someone within the management team decided that RPG needed them too. I considered it a goofy feature, but I was too tired and jaded to argue. (Now that JSON is commonly used in places where XML was previously used, should RPG now include JSON opcodes?)

Lately, the major enhancements to RPG seem to be accompanied by major coverage in the iSeries press. Consider that silly Open Access feature, which seems primarily designed for ISV's helping to modernize old monolithic applications, which in most cases probably should be rewritten from scratch instead. And now with the new free-form specs, a lot of pundits are writing about how the feature will make the language more acceptable to other programmers, while ignoring the functional deficits previously mentioned. That is, these days, it seems like planning RPG content is more of a public relations exercise geared towards managers of client RPG shops rather than providing real improvements making the job of RPG programmers easier. As RPG falls further and further behind the modern programming languages, I suspect the PR show will become more and more prominent.

In my opinion, although I think the new free-form specs are a nice improvement to RPG, they will not make RPG more palatable to the programmers of Java, Python, PHP, etc. A dozen years ago, we saw little justification for the feature, and I don't see what has changed in the meantime. If anything, RPG has become less and less relevant as applications programmers continue to discover the productivity gains to be had by using modern interpreted languages like Python and PHP.