Let's not get physical

In my last post, I promised to reveal how class diagrams can be your secret weapon in the war against imprecise requirements. How can this be, you wonder? Sure, class diagrams capture entities, attributes, and associations, but they're completely incapable of expressing the hopes, fears, aspirations and unwarranted hostility of my end users. Surely we need all the subtlety and nuance of natural language to fully capture the essence of what we're being required to build? Surely we need to capture requirements using the flowing prose of use cases?

As we've discussed, precision is certainly something to be aspired to in requirements specifications, and yet, the legal profession provides compelling testimony to the horrible things that happen to beautiful prose when we impose precision upon it. Let's begin to consider some other things we can express clearly, unambiguously, and some might say beautifully, via everybody's favorite UML diagram.

Suppose we've been asked to build an online storefront offering stolen military hardware at low-low prices. The company, Arms4Less, was founded by several retired mercenaries now holed up somewhere in Honduras. Our primary contact is this crusty old guy we know only as "Sarge". The first thing he says the system needs to do is "store orders for stuff."

"You know", he says, "we sell stuff. Hand grenades, bazookas, napalm cannisters, that kinda thing. We'll need the usual shopping cart deal, letting customers tell us how much of each product they want." We sketch out the diagram below.
As you would expect, an Order consists of 1 or more (1..*) items, and each OrderItem specifies the quantity of some product being ordered. Each Product has a price. Now, hopefully you're regarding this model and saying, "Yeah? So? Why am I wasting time reading this stupid guy's blog when I could be learning Spring?" In a moment we'll make this model a lot more interesting. The diamond, by the way, signifies a "composite" relationship between an Order and its items, which essentially means that if we delete an Order, all of its items will get deleted too. (You would of course know what the diamond means if you'd been paying attention instead of doing email in that UML course your boss sent you to.) If the other symbols being used here are mysterious to you, go hang out with Ambler and come back when you're done.

A tremendously useful, but largely underused (Ambler's overview doesn't even mention them), facility of class diagrams is the specification of "derived" attributes. Derived attribute values are - uh - derived. That is, their values don't need to be stored since they can be completely determined from other data values. For example, Sarge explains that we need to total up each order and make the customer's got enough dough to cover it. "We do this for the customer's own protection", Sarge explains. "You don't wanna know what's gotta happen if the guy can't pay us." And indeed, we really don't wanna know until a later iteration. So, we add 2 derived attributes to the model to deal with the order total.



We've added a total attribute to both the Order and the OrderItem entities. Note that each has a leading slash, which means "derived" in UML. The OrderLine total is, not surprisingly, derived by multiplying the quantity by the price of the associated product (quantity*product.price). The Order total is the sum of the totals of its items (sum(items.total)). We mentioned earlier that derived attributes don't need to be stored, and indeed you can see how we can simply calculate these attributes' values on the fly. Certainly, however, there are situations in which we store derived attributes if, for example, they're too expensive to calculate dynamically. Such a decision however is purely an implementation choice. Logically, these attributes are derived.† This is all about what the system must do, not how it will do it. So let's keep our functional requirements logical. As Sarge might say, "You don't wanna get physical with me unless you're good and ready, computer boy."

Now, you're probably still a little underwhelmed by all this. But as we'll see in our next episode, we've only just scratched the surface of the expressive power of the friendly, little class diagram.

Real programmers don't eat diagrams

Yesterday I discussed how we developers routinely deal with imprecise requirements and while most of us find this to be annoying from time to time, overall we don't consider this a Big Problem. Or perhaps many of us do recognize this as a Big Problem, but have resigned ourselves to its seeming intractability. In any event, we can all agree that if it were possible for us to unambiguously "encode" requirements precisely, and could do this economically, it would surely be better than the current, preferred practice of getting a "pretty good idea" of what some business guy needs, code it up, and then sit down with the guy to see what we got wrong, and come away with a "pretty gooder idea" than we had previously. And then repeating this process over and over again until we get it right or until it's time to go live.

Most development projects begin with a fuzzy expression of requirements as articulated via some combination of text, pictures, and threatening emails by some end users, and end with a concrete expression of requirements as expressed in code and entreaties for forgiveness by programmers. As we discussed, there is currently no widely recognized middle ground between the fuzziness of warm, friendly natural language-based requirements specs, and cold, hard code. As a result, developers are left to interpret the fuzzy as their first step toward implementing a solution. Precision Requirements Modeling (PRM) is a set of techniques for transforming fuzzy requirements into a "code-ready" requirements specification. PRM allows requirements to be expressed with the precision of code, without getting bogged down in requirements details. Perhaps most tantalizingly, PRM specifications are themselves executable, which means we can let end users test drive a specification prior to developers becoming lost within the dense tangle of the implementation jungle, where they become increasingly isolated from civilization as they fashion primitive tools and frameworks, and engage in pitched warfare with competing tribes using .NET.

PRM begins with good-old-fashioned domain modeling. Virtually all O-O developers engage in some level of domain modeling whether they realize it or not. Every time you specify a class that represents a business concept, you're engaging in domain modeling. Even the staunchest of anti-modeling activists can often be caught in moments of weakness drawing UML-ish boxes and lines on a whiteboard. Anybody who has worked on anything but the most trivial of applications has come to recognize the usefulness of UML class diagrams or, at a minimum, entity relationship diagrams as a way of allowing members of a team to get a shared understanding of the major abstractions or data representations being used and how they relate to one another. Of course, you can always just fire up a debugger and trace through your buddy's code, but most of us would prefer a less - uh - extreme approach to integrating the disparate pieces of an implementation.

Class diagrams and ERDs are some of the best ways of communicating among a team an application's entities, their attributes, and their relationships to each other. In terms of attributes, most developers specify no more than names and types of what will be persisted. But as we shall soon see, class diagrams have the potential of communicating so much more if we would just let them. How well do you really know class diagrams? Sure you draw them occasionally. Sometimes you may even bring them to lunch with your colleagues. But if you would just get to really know them, you would see that they're among your best friends.

In our next installment, we'll get better acquainted with some class diagrams and see how, with some minor extensions, they can be used to unambiguously specify large portions of any requirements specification, and bridge the gap between warm, fuzzy business people and cold, hard programmers.

Just what we need -- more lawyers

One of the things you gotta love about "agile" is the name. Is it ever bad to be "agile"? Is there a good alternative (other than synonyms) for "agility"? Here at Gorilla Logic, we've devised what we modestly consider a breakthrough approach to requirements definition. "Precision Requirements Modeling" is the best our marketing department has been able to come up with as the moniker for this new approach. Although mind-numbingly accurate in its characterization, the name lacks a certain zing. Perhaps "Naked Precision Requirements Modeling" would be better.

While all developers want to be rich, thin, agile, and left alone, the same cannot be said of their wanting to be "precise". This is a curious thing when you consider that programming of course is nothing if not "precise". Code specifies precisely what the system will do. For the time being at least, computers don't have the ability to ponder a set of instructions and decide how best to carry them out. Programming code is literal, bereft of simile, metaphor, and clever double entendres. Functional requirements on the other hand are captured as natural language either because they're being written by non-programmers, or because they need to be reviewed and approved by non-programmers. Natural language tends to be pretty imprecise. (That's why we need programming languages.) Our industry has thus become accustomed to routinely working with imprecise requirements specifications.

Now some will argue that natural language can be quite precise. Aren't legal contracts shining examples of precise, natural language? Perhaps the world needs more lawyers. On the other hand, legal contracts tend to be fairly unintelligible to non-lawyers, and although I can't prove this, I will assert that most lawyers would make crummy programmers. Since non-lawyers are going to be writing the code, we probably need something other than legalese to communicate requirements.

Some will say, "aha"! This is precisely why striving for precision in requirements is a waste of time. "Those wacky business people can't possibly know if they want something or not until they see it running on their own desktops anyway". To a large extent this is of course true, but the number of cycles required to get things accepted by an end user will surely be reduced in direct proportion to how well we can understand the requirements prior to writing each round of code. Many theologians will say that although better requirements will reduce the cycles, they don't necessarily reduce the overall effort, since we're just substituting time spent requirements writing for time spent coding (and wouldn't you rather be writing code right now?). In fact, given what we've discussed about the imprecise nature of natural language, aren't we just better off capturing the requirements as code rather than trying to turn a requirements specification into a legal contract?

Perhaps. But this argument ignores the fact that when we code, we deal with far more than functional requirements. As much as 80% of code can be concerned with implementation details as opposed to business logic. (Actually, I just totally made up this statistic, but if you write code, I'm sure you'll agree that a very large proportion of code is solely concerned with plumbing). So the problem is that when we run off and code up a set of loosely defined requirements, we spend a whole lot of time dealing with implementation details that along with the business logic will need to be redone if our interpretation of those requirements is somehow incorrect.

Ah, but isn't this what "refactoring" is all about? The answer to that question is "no". Refactoring is about cleaning up an implementation. It's about avoiding over-engineering by just writing the damn code, and then restructuring that code when it becomes unwieldy, rather than building elaborate frameworks before we truly understand what our implementation requires. "Requirements refactoring" is a euphemism for getting things wrong. When we need to "refactor code" to reflect a requirements "clarification", we are simply rewriting code that was written to do something nobody actually wanted.

If there were only some way to "code" requirements without having to deal with implementation details. Then, we could indeed use code as our medium for capturing precise requirements, without wasting a lot of time on unneeded implementation should those requirements prove inaccurate. Hmmm, if only there were some way to capture requirements as code without implementation details.... Oh, that's right. There is a way. And coincidentally, it's what we've been working on for the last six years here at GL.

Searching for love in all the wrong use cases

Having established (ie, forcefully asserted) that it is generally Bad to engage in ritualistic development practices, but Good to adhere to a particular practice when it's well-suited to surmounting some challenge, let's consider the first snarling beast that leaps out to menace most young, impressionable development projects before they've even secured funding -- requirements definition.

Functional requirements are strange and mysterious things. Many programmers have never actually seen any, relying instead on rumor, innuendo or hearsay to guide their development objectives. For many, functional requirements are the only things keeping programming from being their dream vocation. Programming after all is, at its best, a beautifully artistic endeavor wherein we devise clever structures to manage the complexity of some problem, and transform chaos into order. Implementation "elegance" is regarded as one of the highest achievements we can attain as developers. As each new project begins, we think wistfully of the various forms of abstraction and all the intricate frameworks we have mastered over the years, and consider how, armed with the depth of our experience, this project will surely be different from our past ones. This time we'll build the thing Right, methodically assembling a great edifice from a series of simple building blocks. Each will be as cohesive as the nucleus of an atom, but as loosely coupled from one another as the grains of sand along the seashore. You kick off your shoes, and wander along the beach, smelling the salt air and thinking about that time under the boardwalk -- then, suddenly, a subject matter expert comes marching down the beach with several unruly business owners in tow, and all hell breaks loose.

Real-world requirements are seldom elegant. On the contrary, they tend to be maddeningly messy. For many applications, the hand-waving of some business stakeholder-guy can be sufficient for reaching an understanding of what needs to be built. But, alas, the majority of applications we build are far more "interesting" than these. Confronted by the terrifying unknown of some obscure workflow that has evolved over millions of years and countless mergers and acquisitions, many developers immediately fall back on their religious texts for guidance and comfort against the onrushing horror. They begin frantically writing user stories, drawing anatomically inaccurate stick figures, and in the dark of the night wondering why the gods of order and logic have forsaken them despite their best attempts to follow the good and righteous path documented and indexed in their most venerated religous text (second edition, totally updated with a new preface by the author).

Why must it be so? Why do business requirements, which on the surface seem so simple compared to the complexity of the software platforms we, as accomplished software developers, whip into submission almost every day, so often defy our best efforts to wrestle them to the ground though we pelt them with CRC cards and lock them in use cases? Is there no way to reduce the arbitrary tangle of business rules to some elegant expression of the problem that can return us to our idyllic afternoon at the seaside?

Grab a towel and some sunscreen and allow me to introduce you to Precision Requirements Modeling.

Too Much Religion

Programmers are a religious sort, unquestioningly engaging in ritualistic practices pronounced as "good" by the industry's high priests, and likewise shunning those practices branded as blasphemous. Everyone knows for example that mixing business logic with user interface code will damn your soul to Hell for all of eternity. Likewise, engaging in waterfall development leads to nowhere but poverty and pestilence. And each time you use dependency injection, an angel gets his wings.

The religious tenets of our industry of course have mostly arisen from hard-learned lessons about what seems to generally work and what seems generally not to. Still, it is clearly the case that there is no single path to project nirvana. Every development initiative grapples with a unique variety of technical, political, and sociological challenges. Some projects even succeed. It is of course natural that people examine the practices used by the really successful ones, and strive to emulate them to maximize their own chances for salvation.

Successful software development, alas, cannot be achieved by blindly following the prescriptions laid down in some sacred textbook or blog posting. The sacred texts do indeed provide valuable techniques discovered only after the spilling of vast amounts of blood by our ancestors who came down out of their mainframes to spread out across the network, converting green screens to rich clients, and procedures to objects. So, let us embrace the tools and techniques, but let us also recognize that the reason software projects fail is that there is no "Way" that one can simply follow to guarantee success.

Software development projects present us with a series of challenges. Each of those challenges can generally be addressed with some established set of tools and techniques. When we complete a successful project we are typically tempted to look back on the steps we took along the way and announce to our colleagues, "Ah, yes, truly we have found The Way. Let The Way be written down so that it be made repeatably so, and we can look forward to nothing but happy days for ourselves, our managers, and our manager's managers." And yet we then see others try to follow The Way, only to find misery, chaos, and even death. Okay, maybe not death.

So are we condemned then to roll the dice each time we undertake a new development initiative? I don't think so. But the solution is not religion. The solution is in learning a few highly useful techniques, but rather than applying them ritualistically, we should instead learn to assess the particular challenges presented at a particular juncture of a particular project, and to select among and adapt the various techniques to the challenge at hand.

The Way does not exist. This is The Only Way.