Big Gorilla - Stu Stern's Blog

FoneMonkey for Android Q&A

2011-12-09T12:16:00.002-07:00

I'm interviewed about FoneMonkey for Android: http://jaxenter.com/fonemonkey-for-android-q-a-39195.html.

FoneMonkey, Objective-C, and the Dark Arts

2011-11-21T20:56:00.003-07:00

Dr. Dobbs just published part 2 of my series on FoneMonkey for iOS. Part 2 gets deep into the very mysterious business of how we record and playback user interactions.

Not only does this article reveal the secrets to extending FoneMonkey, but it also reveals techniques for doing unholy things in Objective-C like replacing method implementations and grafting methods onto objects at runtime.

Java took them away, but Objective-C puts the guns and knives back into programming!

FoneMonkey for Android has Landed!

2011-11-14T22:40:00.003-07:00

Just pushed FoneMonkey for Android 0.6 Early Access out the door. You can download it here. We'll be uploading documentation and sample projects over the next few days.

But now...sleep.

Adobe kills Flash

2011-11-09T08:28:00.003-07:00

Given that mobile is the future (and the present), this news that Adobe is EOL'ing mobile Flash amounts to a death sentence for desktop Flash as well. It's sad (and perhaps disturbing) that a mature and powerful technology loved by so many was so easily killed off by the neighborhood bully (and everybody just stood by and watched).

Time to learn to love JavaScript.

Dr. Dobbs meets the monkey

2011-10-25T11:37:00.002-06:00

I think Dr. Dobb's was the first tech mag I ever read (back in the 80's!). The good doctor just published an article I wrote about FoneMonkey.

We're the "E" in iOS: Open source projects fill Enterprise holes in Xcode

2011-05-11T16:54:00.000-06:00

A long, long time ago, just as the internet bubble was really getting going, many pundits were talking about “internet time” to describe the radical time compression brought about by the web. Software release cycles were suddenly occurring over periods of just a few months rather than years, and technology platforms were similarly revving over just a few years whereas previously it had literally taken decades for enterprise IT to make any major changes in how they built software.

If internet time was fast, what are we to make of “mobile time”? The big bang of mobile time, the release of the first iPhone, was just four years ago. Enterprises certainly needed to move quickly to keep up with internet time, but at roughly the same four-year mark, most enterprises were doing little more than creating static websites. There has been no comparable gestation period for mobile development since Apple was nice enough to skip over infancy and adolescence and give birth on day one to fully formed, mature applications employing radically new user interfaces.

While iPhone applications have been pretty “magical” since day one, enterprise-class tool support for iOS app development has been quite a bit slower in coming. It’s hard to find many enterprise developers having anything nice to say about Apple’s Xcode IDE, which is currently the only serious game in town for developing iOS apps. Try the Google search Xcode +”piece of crap” and you’ll be treated to more than 40,000 results. There are of course many similar but more colorful searches you can try.

Back in the days of internet time, Sun created a virtually unusable IDE called Java Workshop. Fortunately for internet time, other companies including IBM, Borland, and Symantec created competing IDE’s and Java development has enjoyed robust tool support ever since. The closed nature of iOS however has understandably dampened the enthusiasm anybody outside of Apple might have for jumping into the nascent iOS development ecosystem. After all, Apple can at any time change the iOS platform in such a way as to make third-party tools incompatible, similar to the way in which iTunes was continually revved to maintain incompatibility with the Palm Pre mobile phone.

So, since the usual commercial suspects have too much business sense to enter into the iOS development ecosystem, it’s left up to those of us with little or no business sense – I am of course referring to open source software developers – to fill the gap!

FoneMonkey 5, baby!

2011-04-04T08:24:00.002-06:00

This morning at 7am we unleashed FoneMonkey 5, our first officially supported production release of our record/playback functional testing tool for native iOS apps on iPhone and iPad.

FoneMonkey 5 is by far our most solid release yet. Go get it now at http://www.gorillalogic.com/fonemonkey!

My Podcast with Coté

2011-04-01T15:32:00.005-06:00

Yesterday I had the pleasure of being a guest on RedMonk analyst Michael Coté's "Make All" podcast. We discussed the evolution of software dev over the last 15 years, and talked about how native apps and client-side runtimes like Flash support the delivery of full-blown client-side user interfaces, which back in 1996 is what most of us expected from Java applets.

Back then, it wasn't long before we woke up to the twin realities of browser compatibility and bandwidth constraints and were forced to shunt Java from the front- to the back-end of application development, and we spent several years dealing with the page-based hack that became known as MVC2.

Plentiful bandwidth and a new generation of client-side technologies such as Flex, Silverlight, Android, and iOS are finally allowing us to build user interfaces in a much more direct, natural, and efficient fashion than the page-based MVC2 approach, and arguably, provide similar advantages over the the JavaScript-meets-DOM hack now known as AJAX.

You can listen to my complete conversation with Michael here.

Gosling@Google

2011-03-31T13:26:00.002-06:00

The web is abuzz today with speculation about just what James Gosling will be doing for his new employer, Google. My own two cents are included in this post from John Waters at Application Development Trends.

FoneMonkey 4.2c improves text input handling

2011-03-04T21:14:00.001-07:00

We very pleased to announce the availability of FoneMonkey 4.2c which improves handling for UITextField and UITextView components.

4.2c provides robust handling of keyboard input, including recording of the Return key, and fixes an issue where sometimes touch events that ended field editing would be recorded prior to recording the text input itself. On playback, FoneMonkey now triggers all appropriate delegate methods and notifications.

We believe that keyboard-related input issues were the last major problem facing us and 4.2c should reliably provide recording and playback for virtually all common user interface gestures.

Script and generated code storage has been moved back to the base Documents directory to make it easier to use iTunes to move scripts on and off iPhone and iPad devices. FoneMonkey scripts are now suffixed with .fm. You will need to rename any existing scripts.

FoneMonkey 4.2c is available for immediate download at www.gorillalogic.com/fonemonkey. Thanks for the continued feedback. Your input is the primary factor determining what we tackle next!

FoneMonkey 4.2 adds iOS UIAutomation Support

2011-02-08T10:53:00.000-07:00

We're excited (especially since now we can finally get some sleep) to announce the availability of FoneMonkey 4.2, the latest incarnation of Gorilla Logic's free and open source functional testing tool for iOS apps on the iPhone and iPad.

FoneMonkey 4.2 includes numerous bug fixes and adds several significant new features including:

Generation of ready-to-run OCUnit test scripts - Automatically convert "native" FoneMonkey test scripts to Objective-C code that can be extended with Objective-C control logic to add control flow or data-driving logic to your tests.
Generation of ready-to-run JavaScript-based tests for Apple's UIAutomation Framework - Automatically convert "native" FoneMonkey scripts to JavaScript code that can be run in Apple's Automation Instrument. You can uyse FoneMonkey to record UIAutomation scripts which allows for test scripts to be maintained in JavaScript rather than Objective-C. Since these UIAutomation scripts require no FoneMonkey libraries to run, they can be executed against "release" builds of your application.
Recording and Playback of Device Rotation - You can record and playback device rotations on the simulator or on devices.

This release is a significant step forward in the maturing of the FoneMonkey project. Thanks to the members of the FoneMonkey community for providing the feedback and support essential to making FoneMonkey a continuing success.

FoneMonkey 4.2 is available for download at http://www.gorillalogic.com/fonemonkey.

Happy testing!

-Stu

See FoneMonkey at iPhone DevCon!

2011-01-18T14:16:00.004-07:00

I'll be presenting Automating iOS User Interface Testing with FoneMonkey at iPhone/iPad DevCon East in April. FoneMonkey is a free and open source, record/playback functional testing tool for native iPhone and iPad apps. If you're going to DevCon, come on by and check out the monkey!

FlexMonkium 4.1.4 Released!

2010-12-16T16:12:00.003-07:00

We are pleased to announce the newest version of FlexMonkium, Gorilla Logic's plugin that adds Flex recording and playback capabilities to the Selenium web testing framework. FlexMonkium 4.1.4, available for download from www.gorillalogic.com/flexmonkey.

FlexMonkium 4.1.4 includes some important enhancements, bug fixes and usability features including:

Support for running FlexMonkey tests in Internet Explorer with Selenium-RC. Previous versions of FlexMonkium only supported Firefox.
Bug fixes for the getFlexMonkeyValue command, which now reliably retrieves values from properties and table cells.
Commands can now be made to wait for a component to appear before executing. For example, a command for clicking on a button will wait for the button to appear before attempting to click it.
Ability to verify (or retrieve) values for property path expressions. For example, you can verify the number of rows in a DataGrid with the path expression "dataProvider.length".
By default, all commands and verifies are now recorded as "WaitFor..." commands, which eliminates the need in most cases to slow or pause script execution to give components time to appear on the display.
The XML recorded by Selenium-IDE for FlexMonkey commands has been simplified to make reading and editing easier.

FlexMonkium 4.1.4 is the successor to FlexMonkium 1.1. We changed the version numbering so that it syncs up with FlexMonkey version numbering (which we in turn changed last year to sync up with Flex version numbering).

To upgrade to FlexMokium 4.1.4, install the new console and download the new zip file. Upgrade the Firefox plugin by opening flexmonkium.xpi in Firefox. You will need to compile your app with the latest automation_monkey swc file. You will additionally need to use the new version of user-extensions.js with Selenium-RC.

Happy testing!

Great In-Depth Guide to FlexMonkey

2010-11-18T16:32:00.003-07:00

Jon Rose has just published an extensive guide explaining the finer points of using FlexMonkey. You can check it out here!

New FoneMonkey for iOS Release!

2010-11-08T22:17:00.002-07:00

We've just released the latest version of FoneMonkey, our free and open source functional testing tool for iOS applications on the iPhone, iPad, and iPod Touch.

You can find it at http://www.gorillalogic.com/fonemonkey.

Happy testing!

See FlexMonkey at Flex Camp Wall Street

2010-11-04T20:39:00.002-06:00

I'll be presenting a session on FlexMonkey at Flex Camp Wall Street on November 15th in New York. Come on by to see the latest version of our open source Flex testing tool and let us know what you'd like to see it do in the future.

See FoneMonkey at 360|iDev

2010-11-01T19:56:00.003-06:00

I'll be presenting a session on our open source FoneMonkey testing tool for iOS applications at next week's 360|iDev conference in Austin, TX.

If you're going to be at the conference, come on by to see how to get the most out of FoneMonkey, and let me know what you'd like to see in future releases.

FlexMonkium is here!

2010-06-09T12:06:00.002-06:00

We are very excited today to announce the availability of FlexMonkium, a plugin that adds FlexMonkey recording and playback to Selenium, the popular, open source, web testing framework.

FlexMonkium provides for the testing of hybrid applications consisting of both Flex and HTML components by seamlessly interleaving the recording and playback of Flex and HTML interactions. Recorded tests can be output as JUnit-based scripts that can be extended with Java-based testing logic, and that can be run from build scripts and continuous integration processes.

You can download FlexMonkium from http://www.gorillalogic.com/flexmonkium.

FoneMonkey - A Closer Look (Clever Tester Managazine)

2010-06-05T11:19:00.003-06:00

Recently I was interviewed by Clever Tester about FoneMonkey. You can check it out here.

FlexMonkey 4 and FlexMonkium (DZone)

2010-06-05T11:14:00.002-06:00

I was recently interviewed by DZone about FlexMonkey 4 and FlexMonkium. You can check it out here.

Flex Developer's Journal: Functional testing of Flex RIA with FlexMonkey 4

2010-05-25T11:30:00.004-06:00

I was recently interviewed by Yakov Fain, editor of Flex Developer's Journal. You can check it out here.

FlexMonkium: Get ready for the 119th element!

2010-05-15T16:20:00.005-06:00

FlexMonkium, the Selenium Plugin that adds FlexMonkey recording and playback to Selenium-IDE and Selenium-RC, is undergoing final testing and packaging in preparation for its forthcoming release.

In this pair of videos smuggled from Gorilla Logic labs, deep beneath the surface of the Earth, we see FlexMonkium interleaving Flex with HTML recording and playback, and generated JUnit test scripts being run with Selenium-RC.

FlexMonkey 4 is here!

2010-05-05T17:41:00.003-06:00

We just posted the new installer and accessories zip to FlexMonkey Project Home. Getting FlexMonkey to work with Flex 4 proved to be a bit more work than we'd anticipated but ultimately we got everything working and the result is a solid release that includes support for all Spark components.

Happily, we now return our attention back to FlexMonkium, our FlexMonkey/Selenium bridge, which is rapidly nearing it's first public release!

FlexMonkey 4 is coming!

2010-04-18T21:29:00.002-06:00

We've just completed testing and anticipate releasing FlexMonkey for Flex 4 before the end of April!

This upcoming release, which we think we're going to call "FlexMonkey 4", works perfectly with Flex 4 and handles all of the new Spark components.

FoneMonkey 0.7.2 Released

2010-04-04T21:18:00.003-06:00

Thanks to the continuing feedback from our rapidly growing community of FoneMonkey users, we've been able to identify and fix several issues and bring you FoneMonkey 0.7.2, available now at FoneMonkey Project Home.

This release fixes various bugs including problems with keyboard and scroll recording.

FoneMonkey 0.7.1 Released

2010-03-16T21:15:00.001-06:00

Thanks to the intrepid souls who have taken the plunge into the first ever public release of FoneMonkey, we got some great feedback from 0.7 users and have just posted a new release, 0.7.1, with several important bug fixes and feature enhancements, notably:

Enhancements:

* WaitFor command - You can cause script execution to wait for views to be created or property values to be set
* "Popup Dialog" handling - You can now script UIAlertViews and UIAlertSheets.
* Better control event handling for UITextFields.

Bug fixes:

* OCUnit runner crash
* UINavigationBar rightItem playback

FoneMonkey 0.7.1 can be downloaded from FoneMonkey Project Home. We can understand if you considered 0.7 to be too early a release to try, but surely you'll feel differently about 0.7.1, which includes not one, but two, decimal points.

In all honesty, FoneMonkey, while not yet bug free, is quite stable and will definitely make your life better, at least in terms of iPhone testing.

Selenium Assertions and waitFors in FlexMonkium

2010-02-28T12:03:00.000-07:00

In our previous video, we saw recording and playback of Flex interactions interleaved with browser interactions within a single Selenium script. Today's video shows how we can execute FlexMonkey Verify commands as assertions or waitFor predicates within a Selenium script.

The video shows a simply script being played back at slow speed. When we then try to play the same script back at fast speed, it fails because the script navigates to the page containing a Flex app, but then attempts to interact with it before the app has finished loading.

What we would like to do is have a way to tell Selenium to wait for some condition that signals the app has finished loading. One way to do this is to have Selenium wait for the appearance of a component within the application's window. We interactively create a FlexMonkey Verify command that tests whether the name field has a blank value. Notice that when we create this command, it is also "recorded" by the Selenium IDE as an assertion, assertFlexVerify.

This assertion will fail if it is executed before the swf has finished loading. However, we can ask Selenium to wait for the condition to become true by changing the assertFlexVerify command to be a waitForFlexVerify instead. Now, when we run the script, it waits for the swf to load before continuing, and then successfullly plays the rest of the script at high speed. We can of course use a similar technique to have a Selenium script wait for other Flex application events, such as data being returned from a query to be displayed in a table.

As you can see, we have all the major mechanisms necessary to very clean integration of FlexMonkey and Selenium. There is some trivial syntactic sugar we will likely sprinkle on what the video shows here, and we've still got some code clean up to do before we'll be ready to release, but we believe all the hard problems have been solved so it won't be much longer before FlexMonkium, the FlexMonkey Plug-In for Selenium, is publicly available!

FlexMonkium: The Movie

2010-02-27T21:52:00.001-07:00

Once again, we bring you a video smuggled at great risk from Gorilla Logic Labs, deep beneath the surface of the Earth.

Last week, on this very blog, the world first learned of FlexMonkium, the FlexMonkey plug-in for Selenium. Now, we are pleased to present this exclusive video of FlexMonkium in action, allowing Selenium IDE to record both browser and Flex application interactions within a single Selenium script.

Although the recorded scenario is seemingly simple, it is actually quite wonderful. Selenium IDE begins recording on a plain html page, and continues recording as we navigate to a page containing a Flex application, and Flex application interactions are recorded to the script. Recording continues as we hit an html link ("Back") outside the Flex application in the browser page. For good measure, we continue recording as we navigate again to the page with the Flex app for further interaction, and then navigate back to the plain html page. All Flex and browser interactions have been recorded to a single Selenium script!

And of course, playback works as well.

Just a few more weeks and we'll be ready to release FlexMonkum! If you're interested in early access, please let us know.

We've released FoneMonkey, our record/play functional tester for iPhone

2010-02-23T21:32:00.000-07:00

We officially launched the FoneMonkey open source project today at http://www.gorillalogic.com/fonemonkey.

If you've been following my recent posts, you know that FoneMonkey is the world's first record/playback functional testing tool for the iPhone. FoneMonkey records user interactions with the iPhone, and replays them while verifying that actual results match expected ones.

Key FoneMonkey features:

High fidelity recording and playback of most iPhone gestures including touches, drags, and shakes.
Integrated script editor and test runner.
Recording and playback on both the simulator and actual iPhone device.
Extensible framework can be customized to provide specialized command recording and playback for custom components.
Simple Objective-API can be used to run FoneMonkey scripts using OCUnit.

With complete source, downloads, documentation, video tutorials, and a user forum available now at FoneMonkey Home, we are fully ready to monkey!

FlexMonkium: FlexMonkey/Selenium Bridge

2010-02-22T17:00:00.000-07:00

Hey, all you Selenium people out there, get ready to monkey!

We've been getting a lot of requests about integrating FlexMonkey with Selenium to provide testing of "hybrid" test scenarios involving both Flex and non-Flex web apps. It is of course common to embed Flex applications within web pages that also contain non-Flex, HTML components, and the combined Flex and non-Flex pieces together produce a single user experience and workflow. Ideally, such hybrid user interface scenarios could be automated within a single test case. In fact, many times the functional dependencies between Flex and non-Flex components make it impossible to test one without the other.

While we've been hearing about many efforts to combine FlexMonkey with Selenium to provide for such hybrid testing, we're not aware of anything having yet been made publicly available, and having just completed development of FoneMonkey (which we're officially announcing tomorrow), and FlexMonkey 1.0GA (which we're releasing in about a week), we've had some time to take a look ourselves at how we might go about integrating Selenium with The Monkey.

After exploring various permutations, we hit upon a very robust approach through which we "slave" FlexMonkey recording and playback beneath Selenium. We have created a bridge, which we are provisionally calling FlexMonkium, that forwards FlexMonkey commands to Selenium during recording, and forwards FlexMonkey commands from Selenium to FlexMonkey during playback. Such FlexMonkey recording and playback is seamlessly interleaved with native Selenium recording and playback to easily produce hybrid testing scenarios. For example, in a single recording session you can click on HTML components and Flex components, even across multiple browser pages, with the resulting FlexMonkey and Selenium commands being recorded into a single Selenium script which (with the aid of the FlexMonkium bridge) can then be played back with any Selenium runner.

We are fairly confident we'll have something ready for early access in a matter of weeks! If you'd like to get your hands on FlexMonkium as soon as possible, contact us here.

FoneMonkey is In the Launch Tube

2010-02-10T18:41:00.000-07:00

Although we're not officially launching for another week or so, the FoneMonkey project site is now live at http://www.gorillalogic.com/fonemonkey. We've published the source, binaries, and complete documentation. The issue management system and discussion board are also open for business.

iPhone developers of the world, get ready to monkey!

Drupal Page Editor Bookmarklet

2010-02-06T16:29:00.000-07:00

We're in the process of putting gorillalogic.com under the Drupal content management system. I find Drupal's admin interface to be unbelievably annoying. What I really want to be able to do is navigate gorillalogic.com in my browser, and have a button I can hit to open the current page in the Drupal editor.

So, I created a bookmarklet for this purpose. Drag the link below to your browser's bookmark bar. When you're browsing a Drupal-managed site, click the bookmark to open the current page in the Drupal editor (assuming of course you have editing privilege for the page).

DrupEdit

Using FoneMonkey With OCUnit

2010-02-03T12:25:00.001-07:00

FoneMonkey, the world's first functional testing automation tool for the iPhone, now provides direct support for running FoneMonkey scripts from OCUnit test suites.

You can download the latest version of FoneMonkey here.

Running Under OCUnit

OCUnit (also known as SenTestingKit) is a unit testing framework for Objective-C that's bundled with the XCode IDE. As a member of the xUnit family of testing frameworks, OCUnit is similar to other popular frameworks such as JUnit (for Java), and FlexUnit (for Adobe Flex).

Using OCUnit, you can combine FoneMonkey- and non-FoneMonkey-based tests into comprehensive test suites that can be invoked interactively, as well as from build scripts, and from continuous integration frameworks such as Hudson or Cruise Control. It is also possible to combine OCUnit tests with non-OCUnit (for example, JUnit) tests, and combine the output so as to provide considated test reporting across all front- and back-end components of your application.

Please note that to run FoneMonkey with OCUnit, you should not create a testing bundle. Instead, you use the FoneMonkeyOCUnit plug-in to launch your OCUnit tests cases, as described later in this section.

Adding OCUnit to Your Project

You add OCUnit to your project much in the same way you add any framework.

Right click on the target that includes FoneMonkey, and select Get Info from the menu. The Target Info dialog will be displayed.
Add a new entry to the Linked Libraries by clicking the add (+) button at the bottom of the dialog window. A selection dialog will open.
Click the Add Other... button. A file selection dialog will open.
Navigate to the /Developer/Library/Frameworks folder. Select SenTestingKit.framework.
Click the Add button.

Adding the FoneMonkeyOCUnit Plug-In

The FoneMonkey distribution includes libFoneMonkeyOCUnit.a which is a static library that extends FoneMonkey with an OCUnit test launcher.

To add the library to your project, right-click on your FoneMonkey testing target and select Get Info. The Target Info dialog will be displayed.
On the Build Tab, scroll down to the Linking section. Add -lFoneMonkeyOCUnit to Other Linker Flags so that it now should look something like this:
```
-ObjC -all_load -lFoneMonkey -lFoneMonkeyOCUnit 
```

Writing a Test Case

You write your test cases for FoneMonkey in the same way you write any OCUnit test. To create a test case, simply subclass SenTestCase and use STAssertions to test for expected results. For more information on OCUnit (SenTestingKit), see Apple's Documentation.

Using the FoneMonkey API, you can run FoneMonkey commands from within your test case, and you use OCUnit Let's look at an example:

#import 
#import

#import "FoneMonkey.h"

@interface SampleTest : SenTestCase

- (void) testSample;

@end

@implementation SampleTest

- (void) testSample {
  NSString* lastResult = [[FoneMonkey sharedMonkey] playAndWait:@"Test1"];
  STAssertNil(lastResult, lastResult);
}

@end

You run a FoneMonkey script from a test case by calling runScript:(NSString*)script, where script is the name of a FoneMonkey script stored in your application's Documents directory. runScript: returns nil if the script runs successfully. For example, in the script above we call:

    NSString* lastResult = [[FoneMonkey sharedMonkey] runScript:@"Test1"];

If the script fails for any reason, runScript: returns the message generated from a failed Verify command, or other error message. You need to explicitly add an assertion to test for a non-nil result, and display the returned message if it's non-nil as follows:

STAssertNil(lastResult, lastResult);

You can of course do more in your test case than simply run a script and test its result. You can include addition programming logic and assertions, and run multiple scripts.

Using the FoneMonkey API, It is also possible to build scripts programatically at run time, rather than running a script previously saved.

Running OCUnit Tests

When you link your application with FoneMonkey and FoneMonkeyOCUnit, FoneMonkey will by default run all SenTestCase subclasses, and OCUnit output will be written to the console.

If you would like to exit your application after the OCUnit tests have run, set FM_ENABLE_AUTOEXIT the environment variable.

You can disable the running of OCUnit tests upon FoneMonkey start up by setting the FM_DISABLE_AUTOSTART environment variable. Omitting the -lFoneMonkeyOCUnit linker flag will also disable the running of any OCUnit tests at startup.

Using the FoneMonkey Console

2010-01-24T19:45:00.000-07:00

Creating and Editing Scripts with the FoneMonkey Console

When you launch an application that has been linked with FoneMonkey (see Setup Guide), the FoneMonkey Console is displayed on top of the application’s window.

To begin recording, touch the Record button. The FoneMonkey console will hide and you can manually test your application. As you interact with it, FoneMonkey records FoneMonkey commands corresponding to each user interface gesture.

If you stop interacting with the application for longer than the timeout interval (by default, 2.5 seconds), the FoneMonkey console will reappear on top of your application window.
If you are not yet done recording, click any where over the application to hide the FoneMonkey console and resume recording.

Touching the Pause button hides the FoneMonkey window and lets you interact with your application without recording. The console will reappear after the timeout interval. Touch anywhere on your application to re-hide the console and continue without recording.
Touch the More button to view the list of recorded commands.

Commands are displayed in the Command List table. The Timeout slider can be used to set the timeout interval anywhere from 2 to 10 seconds. If the console is reappearing more quickly than you want it to while the console is recording or paused, make the timeout longer.

Touch the Play button to begin playback. The console hides and FoneMonkey executes each command in sequence, generating user interface actions corresponding to each command.
By default, FoneMonkey pauses .5 seconds between commands. Sometimes you may need FoneMonkey to wait additional time at certain points in a script. For example, a command might need to wait until an animation finishes running. Use Pause commands (see command reference) to insert pauses between commands during playback.

Editing Recorded Commands

To edit a recorded command, touch the Edit button on the row in the command list to be edited. The Command Editor will be displayed.

Using the command editor, you can change the Command, Component Class, Monkey ID, or any of the associated parameters. When you’re finished editing, touch the Done button.

Inserting and Deleting Commands

Commands can be inserted and deleted using the Insert and Delete toolbar buttons, which display the Insert or Delete buttons in each row of the displayed script.
Saving a Script

To save a script, touch the Save Script button. The Save Script dialog will be displayed.

Enter a name for the script and hit the Save button. Scripts are saved in the application's Documents directory.

Running a Saved Script

To run a saved script, touch the Open Script button. The Saved Script List will open displaying all saved scripts.

To open a script, touch its name in the list. The script will open in the console and can be played and edited.

Recording a New Script

To clear the currently opened script and begin creating a new one, click the Open Script button and then touch the New button displayed above the Saved Script List.

Deleting a Saved Script

To delete a saved script, touch the Open Script button and then click the Edit button displayed above the Saved Script List. You may then delete a saved script by touching the associated Delete Icon.

Working with FoneMonkey Command Scripts

2010-01-22T21:27:00.000-07:00

FoneMonkey plays FoneMonkey commands. For each kind of user interface action there is a corresponding command. Some examples of commands are Touch, Scroll, and Shake. (See the FoneMonkey Command Reference for a complete list).

User interface actions are directed to components. For example, you Touch a UIButton, or Scroll a UITableView to Row 5. FlexMonkey commands identify the action to be performed, and the component to receive the action. Components are identified by a combination of their Object-C class name (or superclass name) and a string identifier called a monkeyID.

All commands are written as:

CommandName ComponentType "MonkeyID" Parameters, ...

where

CommandName identifies the command to be executed.
ComponentType is the Objective-C class name of the component to receive the command. If componentType is omitted, it defaults to UIView (ie, any component).
"MonkeyID" is a quoted string that uniquely identifies which instance of the component's class should receive the command. If there is only one instance of a particular type, monkeyID may be omitted.
Parameters, .... are zero or more command-specific parameter values in CSV format.

By overriding recording and playback methods, it's easy to add your own commands to FoneMonkey or customize existing commands. See the FoneMonkey Extension Guide for more information.

Some Command Examples

Touch the "Done" button:

Touch UIButton "Done"

Touch the PaintingView at coordinate (25, 75):

Touch PaintingView 25, 75

Enter text into the "FirstName" field:

InputText UITextField "FirstName" fred

Identifying Components by ClassName and MonkeyID

As mentioned above, commands correspond to user interface actions, and actions are directed to components. For example, a command might specify to Touch a UIButton. If there is only one UIButton on the screen, FoneMonkey knows which button the command is referring to. If there are multiple buttons, however, we must tell FoneMonkey which one to use. In addition to the component's Objective-C classname, and a unique identifier called a monkeyID that's generated by FoneMonkey extensions for various UIKit classes. For any command, you can specify just a className or a monkeyID, or you can specify both.

Component Identification Examples

If there are multiple buttons on the screen:

Touch UIButton "Done"

If there is just one button on the screen:

Touch UIButton

If threre is just one button on the screen with a monkeyID of "Done":

Understanding MonkeyID's

FoneMonkey's UIView extensions provide the default monkeyID for all components. By default, the monkeyID of a component is the value of its accessbilityLabel property, if one exists, its tag property if it's non-zero, and otherwise FoneMonkey generates a unique identifier as explained below. Many component types provide specialized monkeyID's. For example, UIButton returns its titleLabel.text as its monkeyID, UITextField returns its placeholder value.

See the FoneMonkey Command Reference for a description of the monkeyID's returned by each component type.

If a component provides no monkeyID, FoneMoney generates an identifier by assigning an ordinal to each instance of each class on the screen FoneMonkey generated monkeyID's are prefix with a #-sign.

Examples of Generated MonkeyID's

Touch the first button:

Touch UIButton #0

Touch the second button:

Touch UIButton #1

Validating Tests with the Verify Command

You insert Verify commands into your script to validate during playback that actual results match expected ones. The Verify command has the following syntax:

Verify ClassName "monkeyID" propertyPath, expectedValue

As with all FoneMonkey commands, you can specify either ClassName or monkeyID or both to identify the component.

propertyPath is any valid key path to a property.
expectedValue is the expected value of the property identified by the propertyPath.

Verify Command Examples

Check that the last name label is "Smith"

Verify UILable "Last Name" text, Smith

Check that the button says "Hello"

Verify UIButton titleLabel.text, Hello

If a verify command fails, a message is displayed in the FoneMonkey Command List.

Inserting Pauses into a Script

By default, FoneMonkey pauses .5 seconds between commands during playback. Sometimes you may need a longer delay between commands, for example if an animation needs to finish running before a component is displayed.

You can insert pause commands to allow additional time between the execution of commands. It has the following syntax:

Pause milliseconds

where milliseconds is the number of milliseconds to pause before executing the next command.

The Pause command ignores class name and monkey ID, if any are supplied.

Pause Example

Pause for 1 second:

Pause 1000

FoneMonkey: New Fat Library

2010-01-21T20:50:00.000-07:00

Thanks to some helpful advice from Victor Costan about building fat libraries, the latest FoneMonkey library can be used to build applications for both the iPhone device and simulator. The fat library contains code for both arm7 (iPhone) and i386 (simulator) architectures, so you can reference the same library binary from either your simulator of device builds. Note that the linker will strip out unreferenced code so the convenience of the single library does not come at the expensive of a bloated application binary.

The download is available here.

FoneMonkey: The Update

2010-01-16T14:55:00.000-07:00

This release of FoneMonkey, the world's first and only free and open source testing automation tool for the iPhone, fixes various bugs, adds support for additional Cocoa Touch components, and can be run on an actual iPhone device in addition to the simulator.

The latest FoneMonkey download and release instructions can be found here.

We expect to have the official FoneMonkey project site online later this month!

FoneMonkey: The Download

2009-12-29T16:33:00.000-07:00

Tonight I am releasing FoneMonkey into the wild. Without further ado, here it is:

FoneMonkey.zip

The iPhone does not yet allow installation of user-defined frameworks, so FoneMonkey is disbributed as a static library along with the image files and nibs required by the FoneMonkey user interface. FoneMonkey.zip contians libFoneMonkey.a as well as a bunch of png and xib files.

The zip file does not include the FoneMonkey source code, which we'll be releasing when we officially launch the FoneMonkey open source project in mid-January, 2010 (ie, just a few weeks from now!). At that time we'll be going live with a community forum as well. Until then, please post questions or comments to this blog.

To get started using FoneMonkey, download the zip file and extract the FoneMonkey distribution folder. Then follow the instructions below.

Happy testing! We look forward to your feedback!

Testing an iPhone application with FoneMonkey

In order to test an iPhone application with FoneMonkey, you must first link FoneMonkey with your application as shown in the 2-minute video below. Step-by-step textual instructions are also provided here.

You might need to view the video in a new window.

Open your application's project in xcode.
Duplicate your application's build target by right-clicking on it and selecting Duplicate from the menu. A new target will be created called YourApp copy.
Rename YourApp copy to something like YourAppTest.
Add the downloaded FoneMonkey folder to your project by right-clicking on the project and selecting Add > Existing Files... from the menu. Navigate to the FoneMonkey folder, select it, and click the Add button.
When the dialog box appears, select the Recursively create groups for any added folders option.
In the Add to Targets box, deselect YourApp and select YourAppTest.
Click Add.
Right-click on the YourAppTest build target and select Get Info from the menu.
On the General tab, delete libFoneMonkey.a from the Linked Libraries. You will need to add CoreGraphics.framework and QuartzCore.framework to the Linked Libraries list they aren't already there.
On the Build tab, scroll down to the Linking section and click on the Other Linker Flags setting. When the dialog box appears, enter:

-ObjC -lFoneMonkey -LFoneMonkey/lib -all_load
Dismiss the project Info window.
Right-click on YourAppTest build target and select Clean from the menu.
Right-click on YourAppTest build target again and select Build and Debug from the menu.
Your application should start in the simulator. Immediately after it displays, the FoneMonkey console should drop down over it.
See this post for more info about recording and playing back tests with FoneMonkey.

FoneMonkey: The Movie

2009-12-16T22:21:00.000-07:00

In this video, recently smuggled from Gorilla Logic labs deep beneath the surface of the Earth, we see the revolutionary new open source, iPhone application testing tool FoneMonkey recording and playing back interactions with an application that was cobbled together from Apple's UICatalog and GLPaint sample applications (which together are comprised of most of the iPhone SDK UI Components).

The video demonstrates FoneMonkey recording and playing back various actions including button touches, table scrolling and selection, switches, sliders, text entry, tab selection, finger dragging (in this case, painting), and even phone shaking (which is the gesture that causes GLPaint to erase the current painting).

You might need to view this video in a new window.

As you can see, the monkey can really dance. I hope to post the library binary and installation instructions within the next few days.

FoneMonkey: Open Source, Testing Tool for iPhone Applications

2009-12-13T18:35:00.000-07:00

This blog post is the first announcement to the world of FoneMonkey, an open source iPhone testing tool that I've been developing for the last several months. FoneMonkey automates iPhone application testing by recording and playing back user interactions, and verifying the results. FoneMonkey was inspired by FlexMonkey, the open source Flex testing tool I wrote a little more than a year ago, and which has subsequently been significantly enhanced by my colleagues at Gorilla Logic, especially Eric Owens. Today FlexMonkey has more than 4,000 registered users and an active community.

We plan to release FoneMonkey source and binaries (with attendant fanfare) in early January. Here's a sneak preview of what's coming.

Some Technical Background

FoneMonkey has been more challenging than FlexMonkey to create because the iPhone SDK, unlike the Flex SDK, provides no Automation API. Before I could develop FoneMonkey, I had to develop an automation framework. The resulting FoneMonkey API is simple, extensible and initially supports nearly all native iPhone (ie, Cocoa Touch) components.

Like FlexMonkey, FoneMonkey is designed to be used by both programmers and QA testers. Rather than simply recording and playing back low-level events, FoneMonkey records "semantic" events. In other words, FoneMonkey doesn't record that you touched a pixel at coordinate (125, 210), but instead records that you selected table row 5. The resulting scripts are relatively forgiving with respect to cosmetic UI changes. They can also be composed from scratch by directly specifying sequences of commands. (FoneMonkey Command Guide coming soon....).

Each FoneMonkey command is composed of a command name, a component identifier, and a set of command-specific arguments. For example:

Touch UIButton Send (touches the UIButton labeled "Send")
InputText UITextField, your name, Fred (types "Fred" into the UITextField with a field prompt of "your name")

The Flex Automation API provides an AutomatDelegate corresponding to each Flex UIComponent, and a delegate instance is created for each UIComponent instance created at run-time. Automation delegates subscribe to their component counterparts for all UI events, and translate these low-level keyboard and mouse events into high-level component events. FoneMonkey uses a logically similar mechanism, but is instead implemented with Objective-C categories that extend UI component classes with record and playback logic.

While recording, the FoneMonkey framework monitors all UI events and forwards them to the FoneMonkey category methods that provide recording for each component class. During playback, FoneMonkey sends each command to the playback methods also provided by the class's FoneMonkey extension category.

The FoneMonkey framework provides categories that record and play back events for instances of the UIView class and all UIView subclasses. It is straightforward to create a category for any custom UI class that requires custom recording and playback logic, or to further enhance the recording and playback logic provided out of the box by the FoneMonkey framework. (FoneMonkey Extension Guide coming soon....)

Using FoneMonkey

To test an application, you simply statically link it with the FoneMonkey library. (FoneMonkey Setup Guide coming soon....) Upon starting your application, the FoneMonkey Console "drops down" over your application window, displaying the recording and playback controls at the bottom of the screen.

The Elements application

The Elements application with FoneMonkey controls (at the bottom).

The video below shows FoneMonkey being used to test Apple's "The Elements" sample application.

In the video, we touch the record button to hide the FoneMonkey Console and begin recording our interactions with "The Elements" application. FoneMonkey monitors for periods of inactivity and drops back down over the application if we stop interacting with it for more than the current inactivity timeout setting (by default, 2.5 seconds). Touching the play button replays the recorded script.

You might need to view this video in a new window.

FoneMonkey provides the ability to edit scripts and save them for later replaying.

The FoneMonkey script editor

You can add Verify commands to a script to test the values UI component properties or the values of associated object properties (ie, you can use a key-value encoded key path) . In The Elements sample application, a custom class called AtomicElementView is used to display data for the currently selected element. The AtomicElementView has a property called element that references an instance of the AtomicElement class, which in turn as a property called name. In the Verify command below, we test that the current AtomicElementView's element.name has a value of "Lead".

Adding a Verify command to a script

Script results with failed Verify (in red)

The next video shows how to edit a script, add a verify command, save the modified script, and then replay it.

You might need to view this video in a new window.

Over the next several weeks I'll be continuing to update the doc in preparation for our putative January launch. Stay tuned.

Gorilla Logic gets kosher-er

2009-10-31T11:23:00.000-06:00

Since ICAAN has approved hebrew domain names, I think we need to register gorillalogic.chom.

FlexMonkey 1.0 Beta 2 and the FlexMonkey User Guide are here!

2009-08-17T17:15:00.000-06:00

Announcing the immediate availability of FlexMonkey 1.0 Beta 2! After the very successful Beta 1 launch last month, we are pleased to bring you Beta 2. Since Beta 1 had very frew issues, Beta 2 is much more of an enhancement than a bug-fix release.

Most notably, Beta 2 introduces "MonkeyLink", which allows you to link FlexMonkey directly into your target SWF. If you have had any problems dynamically loading your application through the FlexMonkey Console or the MonkeyAgent, you can now instead compile MonkeyLink into your application SWF, and then launch your application directly. Using MonkeyLink, you can test virtually any Flex or AIR application. Dynamic SWF loading with the Console or the MonkeyAgent are of course still supported so you also continue to have the option of testing SWFs without having to first recompile them.

Download FlexMonkey 1.0 Beta 2 now at http://flexmonkey.gorillalogic.com/gl/stuff.flexmonkey.download.html!

The long-awaited FlexMonkey User Guide is available at http://flexmonkey.gorillalogic.com/gl/flexmonkey/FlexMonkeyUserGuideR1b2.pdf! Weighing in at nearly 60 pages, the FlexMonkey User Guide is like having your very own Eric Owens!

Happy testing!

Who you calling ugly?

2009-07-16T14:27:00.000-06:00

As mentioned in my previous post, the web has been abuzz over FlexMonkey this week. I noticed that artima.com was referencing us and so I clicked on over to see what they had to say. Here's the quote from Bruce Eckel:

Looking closer to home, Jon Rose at Gorilla Logic has just released Flex Monkey, a free tool to automatically test your Flex UIs. I've known Jon through various events so decided to look at the site. As with many sites, it contains a fair amount of cleverness but no clear statement of what they are about. However, among the slogans I found one that I thought had promise: "The answer is not more monkeys." It's succinct, catchy, and on the right track, because it's suggesting that more things should be automated (and Flex Monkey is an example of that kind of automation). Alas, the rest of the site doesn't give me a clear idea of what they do.

Now I've done some hanging with Bruce and he's someone I like and respect, so I talked to our graphic designers and had them make some changes to our home page.

Before

After

In all seriousness, we do appreciate any and all feedback on the site (and we do really like Bruce).

Meet the Monkey

2009-07-16T13:09:00.000-06:00

We're gratified by all the buzz being generating around FlexMonkey in the wake of our 1.0 release this week. A search for FlexMonkey +testing on google returns more than 12,000 links.

My own article, which gives an overview of FlexMonkey's advanced features and discusses user interface testing more generally, was published today on InfoQ at http://www.infoq.com/articles/flexmonkey-ui-unit-testing. Eric Owens, our lead developer on the FlexMonkey project published an intro article on Adobe Devcenter at http://www.adobe.com/devnet/flex/articles/flexmonkey.html.

If you haven't already, please take a minute to meet the monkey and let us know what you think!

The monkey has landed!

2009-07-14T14:06:00.000-06:00

FlexMonkey 1.0 is here! After nearly nine months of gestation we've delivered what is undoubtedly the premier record/playback testing automation tool for Adobe Flex and AIR applications. And it's free!

For the thousands of folks who are already using FlexMonkey 0.8, FlexMonkey 1.0 brings significant new functionality including a completely revamped user interface which makes it much easier to record and playback tests. We've also added the ability to take snapshots of portions of the screen or of property values to be verified during playback. In addition to testing Flex apps, FlexMonkey 1.0 for the first time provides direct support for testing AIR apps. The FlexMonkey Console is now itself an AIR app, but can launch standalone as well as browser-based SWFs for testing.

Please let us show you our monkey at http://flexmonkey.gorillalogic.com! You can also check out Gorilla Logic's Flex service offerings at http://www.gorillalogic.com/what.development.services.flex.html.

In a Perfect World....

2009-06-01T16:08:00.000-06:00

In a perfect world, we could focus solely on implementing functional requirements and not have to do anything explicitly to deal with non-functional constraints such as keeping shared resources from becoming bottlenecks or contending with bandwidth limitations. It can be instructive to consider what an application would look like if there were no physical constraints. Imagine if we lived in a world with infinite memory, infinite processing capacity, unlimited bandwidth, zero down-time, and interesting television shows (Imagine really hard). In such a world, queries are blindingly fast , there is no limit to how often we interact with the back-end, and we can can constantly exchange enormous amounts of data if we need to. With such a platform, we are free to develop our application without any consideration of physical constraints. Each view can refresh the data it is displaying nearly instantaneously, and we can immediately transmit changes to the back-end, and immediately receive notifications of any errors encountered.

In such an environment, the need for a great many design constraints is removed, although not completely eliminated. Even with physical constraints removed, logical constraints remain. Still, removing the physical constraints certainly simplifies software construction by removing the need to manage limited resources efficiently. Of course, in the real world of enterprise applications, there is no such environment, but not every physical or logical constraint is an issue for every application. Low volume applications or those with very small and simple data requirements can often behave almost as if they were executing in the (near-)perfect world. Consider for example an application that allows a user to update their name, address, and credit card information. As soon as the user logs in, you can easily retrieve such a small volume of data immediately, and you wouldn't have to contend with the user's name and address being changed by someone else simulataneously. You can then allow the user to update any of the returned data, updating their name and address, deleting some credit cards and adding others. If there are few simultaneous users, we can transmit each change immediately to the server where we immediately run required edit checks against it and report any errors encountered back to the client for communication to the user via the UI. Such an approach allows us to write an application in what is arguably the most “natural” way, ie, the simplest way. Physical constraints, on the other hand, force us to commit "unnatural acts", ie, they force us to write code that is more complex than it needs to be from a purely logical perspective.

With RIA's, there are a few physical constraints which are fundamental at this point in time. The client application is a separate process (on a unique platform), and all communication with the server-side must be accomplished by messaging. It is this constraint, more than anything else, that distinguishes ERIA architecture from web applications, two-tier client server applications, or one-tier local applications. And speaking of one-tier applications, it is worth noting that virtually no enterprise application allows for the data to be persisted entirely on the client box. It is virtually always the case that we cannot rely on the client itself for safe storage of business records, and there are typically large amounts of non-client-specific data (such as a product catalog) that are usually too large to make client-side distribution practical.

Another constraint is the almost universal requirement that persistence be provided by a non-object-oriented datastore, typically a relational database. There is of course a fundamental impedance mismatch between object- and a relational-centric data structures that is at the heart of many unnatural programming acts, and although there have been many variations on “object databases” over the years, relational databases continue to dominate for good reasons we’ll explore in some depth another time. In a perfect world, queries would return graphs of objects rather than lists of rows. Fortunately, persistence management frameworks such as Hibernate automate a good deal of the acrobatics involved in object-relational mapping, but unfortunately are not (yet?) fully transparent to the application developer, requiring varying amounts of explicit coding.

Coding in the real world requires more effort than coding in the perfect world because we need to add code to work around physical constraints. It would seem to be self-evident that since it takes more effort to add more code, you should only do so when there is an actual requirement to do so. In other words, the default architecture for any ERIA is essentially the simple one described above. We retrieve all the data up-front. Operate on the data as necessary. And send edits to the back-end as they occur. In pondering each of the questions posed above, the default answer in each case is this straight-forward, perfect world architecture since it requires the least effort to build. In each case however it is also necessary to consider how physical constraints can force the undertaking of more complicated strategies, but again, it should be emphasized that in the absence of any such physical constraints, the simplicity of the perfect world approach is the way to go.

What is an architecture and why do you want one?

2009-02-17T21:22:00.000-07:00

An architecture is essentially a set of design constraints imposed on an IT development project. You want one because unconstrained design leads to impedance mismatches among the various pieces of a system due to logical or physical incompatibilities among software components developed by different team members or third parties. The result is software that takes longer or costs more to build, or is more expensive to operate once it's deployed.

Impedance mismatches can manifest in a variety of ways.

Model - Multiple, possibly inconsistent object definitions for the same logical entity
API - Incompatible arguments or return results required or produced by libraries
Performance - Inability of data sources to provide data as fast as required
Locks - Contention resulting from different components having different expectations for how resources should be shared
Transactions - Improperly serialized updates resulting in data being lost or overwritten with stale values
Responsibility - Functionality needlessly duplicated or incorrectly assumed to be provided elsewhere
Recovery - Components in the event of failure not coordinating properly to minimize downtime or data loss
Operational - Software too difficult to modify in response to anticipated future needs

An architecture is sufficiently defined when you can turn a developer loose on developing some piece of functionality and if he or she obeys all the established design constraints, there is a minimal possibility of creating software that produces any of the above problems. If you're conceptually bothered by being constrained, preferring instead to program in wide open spaces with the wind blowing in your hair, perhaps you would prefer to instead think of architecture a form of freedom -- the freedom from choice. Once defined, an architecture frees developers from having to make (possibly incompatible) choices for how to implement an application's functionality, and instead devote most of their time to implementing functional requirements while minimizing the time spent hacking together glue to integrate with the rest of the team's code.

Here's a very high-level breakdown of a typical ERIA:

BackendService

In any non-trivial enterprise application development project, multiple developers will be working in tandem, and in an RIA project where the client and server platforms do not even use the same programming language, it is natural that developers will be divided into front-end and back-end teams. In order to reduce dependencies between client and server code development and allow the teams to work in relative independence on their respective components, it makes sense to explicitly define an API that will provide the linkage between the client and server code.

The BackendService is typically implemented using a webservice-type container, for example Tomcat and associated plugins, which provides a framework for authorizing users, managing session data, and communicating over web protocols such as REST, SOAP, or AMF.

UserSession

As with most traditional web applications, we require a container that maintains session state for each logged in user. Unless application users are anonymous, the session state will at a minimum contain user identity information required for access control. For many applications, code can be simplified by maintaining other session-specific information on the server, such as for example the current contents of the user's shopping cart.

UserIdentity

In most applications, we require user identity information in order to be able to complete requests received from a client. For example, when a user requests information for "my account", the server must need to somehow know which account to access. It is typically not an option to supply the account number for example as part of the request, since the client could be running a hacked application that allows the user to specify an arbitrary account number. When the user authenticates with the server, the server must associate the authorized user's identity with the session, and check each subsequent request against the that identity information in order to perform the requisite access control.

UserIdentity is typically implemented with role-based security frameworks such as JEE security implementations riding on LDAP or some other user directory store.

DomainObject

DomainObjects are instances of classes that directly model some aspect of our problem domain, such as Customers, Invoices, and Products. These objects are mostly created from input received from the client, or by being retrieved from a database or some external source, and additionally provide the non-UI-specific logic comprising our application.[Mention DDD?]

If we do our jobs right, most of the custom code we write on the backend will be domain-specific, having delegated most of the non-domain-specific plumbing to third-party frameworks.

PersistenceManager

The PersistenceManager caches previously retrieved data and collects updates triggered by the client until such time as a logical transaction is completed and can be committed to a database or external system. For relational databases, the PersistenceManager is typically implemented with a framework such as Hibernate. [Link to hibernate definition?]

BackEndServiceAPI

Clients access the BackEndService via a library that provides a client-code-friendly interface. This BackEndServiceAPI is often no more than a wrapper that exposes the BackEndService interface in the client's native language, and does little more than marshal arguments as necessary to call whatever remoting framework is being used for backend communication, and set up whatever callbacks are needed in cases where results are returned asynchronously.

The BackEndServiceAPI would usually be defined as an interface for which a mock implementation can be used as a stand-in so that the front-end team has the option of proceeding with development ahead of the back-end implementation being complete.

DomainDTO

In a typical RIA, we cannot actually return full-blown objects from the server to the client or vice versa. Instead, we can only transmit an object's primitive property values. For example, if the client requests a particular Customer, the server does not return an actual instance of a DomainObject, but instead returns a set of values such as name, address, and account number. While its possible to construct an actual DomainObject instance from these returned values, it is often unnecessary since an RIA in many cases is merely providing for the display and editing of such primitive values, delegating any requisite business logic back to the server-based DomainObjects. DomainDTO's are objects that provide holders for these values, but don't necessarily provide any "behavior" in the form of methods on the objects. It is of course also common to selectively replicate some DomainObject logic on the client, especially simple field edits, and in some cases we might even replicate complex logic on the client, so we're using the term DTO loosely here. [Link to DTO definition?]

[Diagram showing a DomainObject vs corresponding DomainDTO?]

CacheManager

With a thin client, client-side state does not usually consist of anymore than the contents of some html form or table. Each new view usually requires a round-trip to the server to retrieve both the definition of the view (ie, an html page), and the data presented within it. Even just changing the sort sequence of the rows in a table require re-retrieving a fully formatted table of data from a server. One of the big advantages in an RIA is that we can provide a much more interactive and responsive user experience by not requiring the constant back-and-forth of data and markup with the server. Instead we can retrieve (possibly large amounts of) data just once, display it in a variety of ways, allow the user to edit it, and send updates to the server only after a transaction is logically complete.

The CacheManager keeps track of what we've already retrieved from the server so that we don't incur unnecessary overhead of re-retrieving the same data, and tracks what objects are changed ("dirty") and require transmission back to the server. The sophistication of CacheManagers can vary greatly from one application to another, consisting of little more than hash tables of objects by type [Include a simple diagram of this?], or as much as client-side relational databases.

ViewComponent

The view components are the widgets that provide for the display and editing data, usually DomainDTO's retrieved from the local cache. By ViewComponents, we don't mean low-level UI components like buttons and textfields, but semantically rich composite components such as for example a Customer Creation Form or an Order History Table.

EventBus

RIA platforms are event-driven user interface frameworks where ViewComponents emit and respond to asynchronous notifications such as "User Wants to Quit" or "User Has Updated His Credit Card Info". While such event-driven models are ideally suited to coordinating the asynchronous update and display of data across possibly multiple views, event-driven programs can degenerate into a tangle of interdependencies among components dispatching and listening for each other's events. An EventBus organizes event handling around a hub-and-spoke where events are logically broadcast throughout the application. In this way, brittle point-to-point connections are replaced by a robust, centralized switchboard for application events.

Application

The Application is "everything else". It's the code necessary to integrate the BackEndAPI, the ViewComponents, and the CacheManager into the functioning whole experienced by the user. The Application for example binds particular views to particular DTO's, and invokes particular BackEndAPI methods in response to user interactions.

Can I just go build my application now?

Having dilligently studied the above description, you may be chomping at the bit to put the above principles to work and begin building your next RIA. Unfortunately, the above is not the answer. It only frames the questions more clearly. Chiefly:

Whether to locate logic on the client or server?
How much data to cache locally on the client?
When should data be transmitted from the client to the server?
When should data be persisted to a database?
How should view components be made aware of updates to what is being displayed?
What is the right level of granularity for remote operations?

What's that you say? You came here for answers? Fear not. All will be revealed. But not this afternoon....

The Year of Coding Dangerously

2009-01-27T19:57:00.000-07:00

You've got an appointment set up with your customer at a lunch place down by the water. You get there early, sit down at a booth and check the place out. There's a beautiful woman sitting at a table near the door, watching a tiny movie on her iPhone. You think, where have I seen her before? Is she from a rival consulting outfit? Could it be you're being followed? No, no, you think. Just paranoia. On the road too long.

Your customer arrives and sits down heavily on the other side of the table. He pushes a sky blue folder toward you and orders a patty melt. Inside you find a PowerPoint deck and flip through it, amazed that people still use such cheesy clipart. After a minute you look up, and trying to control your voice manage to say, "We can't do it in time." You reach for a cigarette but then realize you don't smoke. So instead you pick up a french fry.

Your customer gets an impatient look on his face and says, "Look, negotiations broke down in Helsinki. It was bad. They quit the consortium. They're not going with the standard. We've gotta do it their way. How bad could their proprietary stuff really be?".

Just then, an explosion rips through the building. You find yourself lying face up in the parking lot just as a helicopter lands and two guys wearing flak jackets with your company's logo jump out and pull you inside. As you fly off, you see the entire island being consumed in a volcanic eruption and sinking slowly into the sea. Just before you pass out again you think, next time I gotta get a local gig.

I'm sure we've all had experiences like this one. As application developers, we live in a world of excitement, intrigue and suspense. Yes, danger lurks around every turn. Sometimes it arrives with a whisper. A never-before-seen error message appears on a console, or a process turns up dead. Other times it comes with a bang. A load balancer goes beserk and the Big Board in the call center lights up like a pinball machine. The lives of thousands of active sessions hang by a thread. Your cell phone rings. The president needs an update....

As asserted in our last post, architecture can be seen as a a set of implementation constraints selected to mitigate risk. There are other ways to think about architecture, but this definition works well in that it emphasizes the "why" rather than the "what" of architecture. By focusing on the "why", we can better determine the "how much" and define no more architecture than necessary to deal with the risk profile of a particular project.

Architecture alone is of course insufficient for mitigating every kind of risk we might encounter on a project. We also need things like legal contracts, customer expectation management, and adequate QA testing, to name a few. Most of those other things are not, strictly speaking, technical risks, and so are not typically the province of application developers, being handled instead by people like lawyers, project managers, and that really annoying guy in the purchasing department.

In defining an Enterprise RIA Architecture, we will begin by identifying the most common risks faced by most development efforts and define our architecture in the context of providing mitigation strategies for those risks.

To be clear, what we mean by "Enterprise RIA" (ERIA) is any application in which a rich client user interface accesses a set of back-end services to execute transactions. ERIA's are characterized much more by this back-end interaction requirement than by any particular attributes of the user interface itself. Because the user interface of an ERIA can vary so dramatically from application to appllication, there's not much we'll say about the architecture of the presentation portion of an application, which might consist of anything from simple forms to sophisticated 3D aimations. We are much more concerned with how the presentation portion handles information and transactions in concert with back-end systems, where the back-end systems are information- or transaction-centric.

What, you may wonder, makes back-end interaction in an RIA fundamentally different from good-old-fashioned J2EE architecture? The answer is that in a typical J2EE application, virtually all the logic lives in the back-end. Yes, we can use JavaScript to execute logic on the client platform directly, but since, by definition, non-RIA applications provide non-rich UI functionality, that logic is typically limited and simple.

In an RIA, however, we are building full-blown client-side applications with client-side logic that can be quite complex. RIA clients tend to be highly stateful, oftentimes maintaining relatively large amounts of data client-side that create unique problems around synchronization with back-end systems of record. As we shall see, such front-end to back-end state synchronization is one of the major technical requirements that drive many of our architectural decisions.

In our next installment, we'll look more closely at the risks confronting most ERIA projects. Until then, watch your back-end and for godsakes make sure you're not being followed by agents of rival consulting outfits....

Architecture, smarchitecture.

2009-01-21T20:50:00.001-07:00

Now begins our discussion of Rich Internet Application Architecture. This is a work in progress, which is to say that what is below is really a draft of a post, but we're exposing the salami-making process to invite collaboration from our friends and lovers.

The first thing we need to discuss is what we mean by "architecture", since this is surely an overloaded term in our industry. It seems to mostly say something about how a system's functionality (domain-specific and otherwise) is partitioned both logically and physically across application code, software platforms, and physical hosts. For the purposes of this discussion we'll define architecture as any set of implementation constraints imposed on a team during some project.

By this definition, a project without a pre-defined architecture is one in which their might be a "design", for example consisting of major classes and how they interact with each other, and choices about what platforms things will run on, but beyond that it is left up to each developer to determine how to implement each piece of functionality assigned to their work queue. We say that the project has no pre-defined architecture because one will almost inevitably "emerge" as various mechanisms and partitioning strategies get worked out and discussed among the team, and people naturally clone already working code when they need to do something similar. It could be argued that the resulting "architecture" is one that is tightly adapted to meet the actual (emerging) needs for one, rather than something possibly over-engineered beyond what we find is really needed after we've been coding on the project for a while.

Of course the real choice is not a binary one between having a pre-defined architecture and not having one. We can rigorously define some aspects of a system's architecture while letting other aspects emerge. But that still leaves the question: "How much architecture do we need?"

I propose that the answer to that question should be "Just enough to mitigate risk" and in summary will close with the proposition that:

A system's architecture definition should consist of the minimal set of implementation constraints needed to mitigate implementation risks.

In our next installment, we'll discuss the most commons risks confronting Enterprise RIA system implementations.

A sketch of a Rich Internet Application Architecture

2009-01-15T15:45:00.001-07:00

Just another code-slinging CEO

2009-01-14T11:41:00.000-07:00

I am a fossil, or more accurately, I should say that I'm a fossil record. Or perhaps the better metaphor would be that I'm like one of those ice cores geologists drill from the depths of the Arctic. If you were to sink a drill deep into my head (and I know many of my former colleagues would like to), you would find evidence of the many fads, trends, and revolutions that have constantly reshaped corporate IT over the past 30 years recorded in my brain like the stratified deposits within an ice core. By examining such ice cores, geologists arrive at a deeper understanding of our planet, and by examining the ice core in my brain, I will endeavor for us to arrive at a deeper understanding of Planet IT. I have not only been present for the many tectonic shifts that periodically rock our industry, I have usually been standing directly on the fault line, intimately involved as a leader, manager, and practitioner of application development.

I want to emphasize the "practioner" part of what I just said. Years ago, my colleagues and I would tell each other (half-) jokingly, "Don't trust anybody who doesn't log on". This was shorthand for our belief (which I still hold) that anybody who no longer understands the technology is inherently ineffective in directing IT initiatives. This does not mean that management needs to write code, but management does need to understand a fairly large body of key technical principles since these significantly impact the planning and execution of any IT project.

I myself have continued to write code in spite of having spent many years in fairly senior management positions (at Sun, for example, I managed 300 people). Hopefully as I delve into the subtleties of Rich Internet Application Architecture in the following series of posts, you will trust that my views are derived not just from a consideration of battles viewed from the safety of an underground bunker back at central command, but also while engaged in fierce trench warfare myself.

This experience, coupled with the ice core in my brain that informs my views with a deep (and painful) appreciation of all that has gone (wrong) before hopefully convinces you that my insights are more valuable than what you would find in a random blog post. (And yes, I realize that this is itself a random blog post and my last comment was a referential recursion of sorts that probably blew the stack of several unsuspecting readers).

In any event, I do hope you'll join me in my next few posts for an exploration of Rich Internet Application Architecture (I hesitate to call this RIAA since it's the RIAA that sues people for downloading Britney Spears singles), but before we begin, let's all go write some code.

The Tower of Babbage

2009-01-08T21:12:00.001-07:00

In this post about an Eric Evans presentation, Jon Rose mentions how Eric needed to clarify that the intention is not for Ubiquitous Languages to be enterprise-wide. UL's are established across project teams, not organizations. I was initially surprised that such a clarification would be necessary since this seems obvious, but then I remembered life in the late eighties....

Love Shack was a smash hit and Enterprise Data Models were all the rage.

Much like the Arthur Clarke science fiction story in which a monastery of monks fulfills the purpose of the universe by recording with a computer every known name of God (when they were finished "overhead, without any fuss, the stars were going out"), the idea was that if we could just catalog EVERY data entity, attribute, and association across the ENTIRE enterprise, then surely we would come to know our domain, our applications, our users, indeed our very inner souls better, and thus build better software faster since all of the various warring factions in IT would finally speak the one, true tongue.

At my company, PaineWebber, several monkish DBA's undertook this task for several years, compiling an ever growing glossary of ever greater weight and size. One night in a dream (or perhaps it was after a series of lunches with a leggy sales rep) our CIO realized he could accelerate our rendevous with destiny by buying somebody else's financial services datamodel. And so it was that for the low, low price of $1.5 million, we purchased a great many binders from First Boston containing a great multitude of boxes and lines.

From time to time, developers would come to seek the truth of the binders. Like astrologers poring over charts of the stars, they would look for signs in the many boxes and lines, looking for clues to unlock the secrets of their particular problem domain. But, alas, while many of the boxes and lines bore striking resemblances to actual people, places, and things from the known world, there would also be striking differences from the reality they knew, and in any case, the detail of the models was simply too overwhelming -- or perhaps it was just too magnificent.

And so we went on as before, speaking our own local dialects and doing our best to communicate with neighboring tribes, using extract files like smoke signals, reeking with the smell of EBCDIC.

At least the stars didn't wink out.

Moe, Larry, and Curly Braces

2008-12-11T15:15:00.000-07:00

We've thus far discussed several of the primary pillars of Precision Modeling -- derivation expressions, conditional expressions, query expressions, and derived associations -- and we've used these to help out our hypothetical (and some might say homicidal) customer, arms4less.com ("where you can get regime change for small change!"). We've been specifying the business rule stating that customers' can pay a surcharge to prevent their enemies from purchasing any of the same weapons they're purchasing themselves. We did this by creating a derived association, enemyProducts, that contains the list of all products we are unable to sell to a particular customer, because they have already been purchased by enemies of the customer who have opted to pay the surcharge:

/enemyProducts : Product = enemies.orders.items[isExclusive].product

Now we just need to somehow use this association to prevent a customer from actually buying any of these enemyProducts. As with the many wonders of Precision Modeling we've previously discussed, we can specify such a requirement precisely and concisely (Hey, that rhymes. Our marketing people will be so pleased.) via a little Gorilla UML.

Notice the constraint expression we've attached to OrderItem:

{not product in order.customer.enemyProducts}

In UML, constraints are specified inside curly braces (many have postulated this proves once and for all that Grady Booch is a die-hard Three Stooges fan). Because this constraint is attached to OrderItem, it is specifying that any product associated with an OrderItem cannot also be included in the list of the order's customer's enemyProduct's list.

Our SME, Sarge, inquires as to how we plan to enforce this constraint? "If it gets violated, there needs to be -- you know -- consquences." We ensure him that we will be sure to thoroughly test our implementation of this constraint to ensure that the software indeed enforces it. Sarge sighs wistully as he removes his hand from a bulge beneath his flack jacket.

The enemy of my enemy is my customer

2008-12-04T13:01:00.000-07:00

If you're new here, this post is part of a more-or-less continuous train-of-thought that begins here.

Today we continue looking at query expressions, which I introduced with much hullabaloo last time, but have thus far only given you the smallest taste. The glutinous among you are surely clamoring for more. So let's eat. But please stop using that use case diagram as a napkin. (Javadoc is much more absorbent).

Sarge has for the first time invited us to the headquarters of Arms4Less.com. After our blindfolds have been removed and our hands untied, we find ourselves seated around a large conference table in a windowless room. Charts around the wall show sales projections and troop strength estimates for the various markets Arms4Less is targeting over the next twelve months.

Last time we looked at how we can derive a value (isTrusted) using a query expression. Let's consider another such example. Sarge explains to us that Arms4Less prides itself on its high ethical standards. "Like if we sell a particular model of tank to one guy, for a small surcharge of 10%, we won't sell the same kinda tank to any of his enemies." We consider this requirement and realize that we need to prevent a customer from ordering any product that has been previously ordered by any of the customer's enemies. Now many of you are probably wonder how the heck we can express something like this on a lowly class diagram. Doesn't this require a more procedurally-oriented specification? Something with a little more get-up-and-go than a class diagram? Class diagrams just lay there, you say. We need something that moves, don't we?

Ha! I laugh at you (with all due respect of course). Do you think I would have wasted this much time blogging about this crap if we could go no further? Consider our latest version of the precision model, below:

We have added a new association, enemies, to keep track of a customer's enemies, and we have added a derived association called enemyProducts that references all the products purchased by a customer's enemies. Let me clarify a few things about "derived associations". First, you may be scratching your head about why we're calling enemyProducts an association at all, since it's not represented by a line on our class diagram. Notice however the type of the enemyProducts attribute. It's a Product. (In fact, as we'll discuss another time, it's a list of Products.)

There are multiple ways of representing associations in UML. For example, the following two representations are virtually equivalent.

On top, we see that the Employee has a BattleInjury reference called injuries. We see exactly the same thing with the Employee representation on the bottom, which contains a BattleInjury reference as well. The only difference is the notation being used. On the top we depict the reference with a composite association named injuries, whereas on the bottom, we decpict the reference with a nested attribute, also called injuries.

A derived association then is simply a derived attribute whose formula evaluates to a list of entities of some type. Let's look at the derivation expression for enemyProducts:

enemies.orders.items[isExclusive].product

In evaluating the value of this expression for a particular customer, we navigate through the customer's associated enemies, to their orders and then to the order's associated items. We then subset (query) the orders to select out those for which enemies have opted for an exclusive deal, and then we navigate from this resulting set to their associated products. The resulting list of products are the ones that we can't sell to the Customer.

The astute reader (or at least most who weren't out drinking last night) will have noticed we're essentially navigating multiple paths in parallel when we navigate enemies.orders.items since a customer can have many enemies, each in turn having many orders, and each of those orders having many items. The expression "unions" the results of navigating these parallel paths.

Now, you may recall earlier in this tirade we talked about the importance of precision. In fact, recall that all these techniques we're using comprise a methodology called Precision Modeling. You can rest assured that I'm not just whipping out arbitrary syntax for each of the formulas we've used. All of them are expressed with a formal expression language that are part of what we call Gorilla UML (GUML).

GUML not only has formal syntax definition. It executes! (Sarge says, "Cool! I execute too!") More on execution later. In our next little visit together, we'll actually look at using GUML to specify constraints. Using constraints we can complete our specification of Sarge's business rule preventing a product being sold to a customer's enemies.

The Axis of e-Ville

2008-12-02T13:40:00.000-07:00

Having looked at how we can use derived and conditional expressions in our class diagrams, we are now ready to consider a construct so powerful that I shudder to think of what could happen if Iran, North Korea, or Microsoft should get hold of it. In fact, I'm taking a huge risk in revealing it here today. If this is my last post, it's a good bet that I'm in the hands of foreign agents trying to force me into assisting with a diabolical plot involving germs, radiation, and Tomcat 5.5.

Yes, today you are ready to learn about (cue some suitably momentous music) query expressions! Wait! Where are you going? Come back. I'm tellin' ya. Query expressions are some serious mojo!

Let us feel their power together by returning again to our consideration of the Arms4Less online superstore ("where you don't need to spend a lot to destroy a lot"). Sarge explains to us that they don't sell to just anybody. "For the really good stuff, you gotta be somebody we trust -- either somebody we know, or a friend of at least two people we know." We easily capture this notion of who's trusted as follows.

We add a Customer entity with an association, friends, referencing customers who are a customer's friends. We give Customer a boolean attribute, isSomebodyWeKnow, and as well as a derived attribute, isTrusted. The value of isTrusted is derived from the expression:

isSomebodyWeKnow or #friends[isSomebodyWeKnow] >= 2

The []-brackets enclose a selection expression. friends[isSomebodyWeKnow] selects all of the friends of some customer where isSomebodyWeKnow is true. The #-sign is a shorthand for "count". So, we can see that isTrusted will evaluate to true if the customer is somebody we know or if they have at least 2 friends we know.

When next we meet, we'll delve into the notion of query expressions more fully. In the meantime, beware of North Koreans toting pirated UML editors....

Conditional Love

2008-12-01T19:35:00.000-07:00

Let's consider some other amusing things we can do with class diagrams. With a little tape, some string, and a pair of scissors, we can of course convert most class diagrams into a ceremonial tribal mask or a pirate hat. Another fun thing we can do is derive attribute values using conditional expressions.

Let's return to our example of the Arms4Less online superstore. We're having lunch with our SME, Sarge, and he's just finished telling us a funny story about a coup he helped stage a few years back. "So Johnny turns to me and says, whadaya mean, where's the king? Who do we got tied up in the back of your car?" We laugh appreciatively, and then turn the conversation back to the requirements for the store. Sarge explains how they want to offer various sales incentives, "like 20% off for quantities over 50". We can easily modify our model to reflect this requirement, by adding a new derived attribute called discount to OrderItem, and then modifying the derivation expression on total to reflect any discounting to be applied. We must also add the necessary non-derived attributes, discountQty and discount, to the Product, as shown.

Hopefully, you are beginning to see how much more you could be getting out of your class diagrams if you would just spend a little more time drawing pictures and a little less time writing code. I know how that can be tough with all the pressure you're getting from religious eXtremists, but as we shall soon see, Precision Requirements Modeling fits very nicely into an Agile approach that even the religious right can love.

In our next installment we'll really get this party started when we consider how to derive values from query expressions. I know the anticipation will be killing you between now and then, and I'd love to show you right now, but I gotta go write some code.

Let's not get physical

2008-11-18T20:24:00.000-07:00

In my last post, I promised to reveal how class diagrams can be your secret weapon in the war against imprecise requirements. How can this be, you wonder? Sure, class diagrams capture entities, attributes, and associations, but they're completely incapable of expressing the hopes, fears, aspirations and unwarranted hostility of my end users. Surely we need all the subtlety and nuance of natural language to fully capture the essence of what we're being required to build? Surely we need to capture requirements using the flowing prose of use cases?

As we've discussed, precision is certainly something to be aspired to in requirements specifications, and yet, the legal profession provides compelling testimony to the horrible things that happen to beautiful prose when we impose precision upon it. Let's begin to consider some other things we can express clearly, unambiguously, and some might say beautifully, via everybody's favorite UML diagram.

Suppose we've been asked to build an online storefront offering stolen military hardware at low-low prices. The company, Arms4Less, was founded by several retired mercenaries now holed up somewhere in Honduras. Our primary contact is this crusty old guy we know only as "Sarge". The first thing he says the system needs to do is "store orders for stuff."

"You know", he says, "we sell stuff. Hand grenades, bazookas, napalm cannisters, that kinda thing. We'll need the usual shopping cart deal, letting customers tell us how much of each product they want." We sketch out the diagram below.
As you would expect, an Order consists of 1 or more (1..*) items, and each OrderItem specifies the quantity of some product being ordered. Each Product has a price. Now, hopefully you're regarding this model and saying, "Yeah? So? Why am I wasting time reading this stupid guy's blog when I could be learning Spring?" In a moment we'll make this model a lot more interesting. The diamond, by the way, signifies a "composite" relationship between an Order and its items, which essentially means that if we delete an Order, all of its items will get deleted too. (You would of course know what the diamond means if you'd been paying attention instead of doing email in that UML course your boss sent you to.) If the other symbols being used here are mysterious to you, go hang out with Ambler and come back when you're done.

A tremendously useful, but largely underused (Ambler's overview doesn't even mention them), facility of class diagrams is the specification of "derived" attributes. Derived attribute values are - uh - derived. That is, their values don't need to be stored since they can be completely determined from other data values. For example, Sarge explains that we need to total up each order and make the customer's got enough dough to cover it. "We do this for the customer's own protection", Sarge explains. "You don't wanna know what's gotta happen if the guy can't pay us." And indeed, we really don't wanna know until a later iteration. So, we add 2 derived attributes to the model to deal with the order total.

We've added a total attribute to both the Order and the OrderItem entities. Note that each has a leading slash, which means "derived" in UML. The OrderLine total is, not surprisingly, derived by multiplying the quantity by the price of the associated product (quantity*product.price). The Order total is the sum of the totals of its items (sum(items.total)). We mentioned earlier that derived attributes don't need to be stored, and indeed you can see how we can simply calculate these attributes' values on the fly. Certainly, however, there are situations in which we store derived attributes if, for example, they're too expensive to calculate dynamically. Such a decision however is purely an implementation choice. Logically, these attributes are derived.† This is all about what the system must do, not how it will do it. So let's keep our functional requirements logical. As Sarge might say, "You don't wanna get physical with me unless you're good and ready, computer boy."

Now, you're probably still a little underwhelmed by all this. But as we'll see in our next episode, we've only just scratched the surface of the expressive power of the friendly, little class diagram.

Real programmers don't eat diagrams

2008-11-17T10:52:00.000-07:00

Yesterday I discussed how we developers routinely deal with imprecise requirements and while most of us find this to be annoying from time to time, overall we don't consider this a Big Problem. Or perhaps many of us do recognize this as a Big Problem, but have resigned ourselves to its seeming intractability. In any event, we can all agree that if it were possible for us to unambiguously "encode" requirements precisely, and could do this economically, it would surely be better than the current, preferred practice of getting a "pretty good idea" of what some business guy needs, code it up, and then sit down with the guy to see what we got wrong, and come away with a "pretty gooder idea" than we had previously. And then repeating this process over and over again until we get it right or until it's time to go live.

Most development projects begin with a fuzzy expression of requirements as articulated via some combination of text, pictures, and threatening emails by some end users, and end with a concrete expression of requirements as expressed in code and entreaties for forgiveness by programmers. As we discussed, there is currently no widely recognized middle ground between the fuzziness of warm, friendly natural language-based requirements specs, and cold, hard code. As a result, developers are left to interpret the fuzzy as their first step toward implementing a solution. Precision Requirements Modeling (PRM) is a set of techniques for transforming fuzzy requirements into a "code-ready" requirements specification. PRM allows requirements to be expressed with the precision of code, without getting bogged down in requirements details. Perhaps most tantalizingly, PRM specifications are themselves executable, which means we can let end users test drive a specification prior to developers becoming lost within the dense tangle of the implementation jungle, where they become increasingly isolated from civilization as they fashion primitive tools and frameworks, and engage in pitched warfare with competing tribes using .NET.

PRM begins with good-old-fashioned domain modeling. Virtually all O-O developers engage in some level of domain modeling whether they realize it or not. Every time you specify a class that represents a business concept, you're engaging in domain modeling. Even the staunchest of anti-modeling activists can often be caught in moments of weakness drawing UML-ish boxes and lines on a whiteboard. Anybody who has worked on anything but the most trivial of applications has come to recognize the usefulness of UML class diagrams or, at a minimum, entity relationship diagrams as a way of allowing members of a team to get a shared understanding of the major abstractions or data representations being used and how they relate to one another. Of course, you can always just fire up a debugger and trace through your buddy's code, but most of us would prefer a less - uh - extreme approach to integrating the disparate pieces of an implementation.

Class diagrams and ERDs are some of the best ways of communicating among a team an application's entities, their attributes, and their relationships to each other. In terms of attributes, most developers specify no more than names and types of what will be persisted. But as we shall soon see, class diagrams have the potential of communicating so much more if we would just let them. How well do you really know class diagrams? Sure you draw them occasionally. Sometimes you may even bring them to lunch with your colleagues. But if you would just get to really know them, you would see that they're among your best friends.

In our next installment, we'll get better acquainted with some class diagrams and see how, with some minor extensions, they can be used to unambiguously specify large portions of any requirements specification, and bridge the gap between warm, fuzzy business people and cold, hard programmers.

Just what we need -- more lawyers

2008-11-16T15:00:00.000-07:00

One of the things you gotta love about "agile" is the name. Is it ever bad to be "agile"? Is there a good alternative (other than synonyms) for "agility"? Here at Gorilla Logic, we've devised what we modestly consider a breakthrough approach to requirements definition. "Precision Requirements Modeling" is the best our marketing department has been able to come up with as the moniker for this new approach. Although mind-numbingly accurate in its characterization, the name lacks a certain zing. Perhaps "Naked Precision Requirements Modeling" would be better.

While all developers want to be rich, thin, agile, and left alone, the same cannot be said of their wanting to be "precise". This is a curious thing when you consider that programming of course is nothing if not "precise". Code specifies precisely what the system will do. For the time being at least, computers don't have the ability to ponder a set of instructions and decide how best to carry them out. Programming code is literal, bereft of simile, metaphor, and clever double entendres. Functional requirements on the other hand are captured as natural language either because they're being written by non-programmers, or because they need to be reviewed and approved by non-programmers. Natural language tends to be pretty imprecise. (That's why we need programming languages.) Our industry has thus become accustomed to routinely working with imprecise requirements specifications.

Now some will argue that natural language can be quite precise. Aren't legal contracts shining examples of precise, natural language? Perhaps the world needs more lawyers. On the other hand, legal contracts tend to be fairly unintelligible to non-lawyers, and although I can't prove this, I will assert that most lawyers would make crummy programmers. Since non-lawyers are going to be writing the code, we probably need something other than legalese to communicate requirements.

Some will say, "aha"! This is precisely why striving for precision in requirements is a waste of time. "Those wacky business people can't possibly know if they want something or not until they see it running on their own desktops anyway". To a large extent this is of course true, but the number of cycles required to get things accepted by an end user will surely be reduced in direct proportion to how well we can understand the requirements prior to writing each round of code. Many theologians will say that although better requirements will reduce the cycles, they don't necessarily reduce the overall effort, since we're just substituting time spent requirements writing for time spent coding (and wouldn't you rather be writing code right now?). In fact, given what we've discussed about the imprecise nature of natural language, aren't we just better off capturing the requirements as code rather than trying to turn a requirements specification into a legal contract?

Perhaps. But this argument ignores the fact that when we code, we deal with far more than functional requirements. As much as 80% of code can be concerned with implementation details as opposed to business logic. (Actually, I just totally made up this statistic, but if you write code, I'm sure you'll agree that a very large proportion of code is solely concerned with plumbing). So the problem is that when we run off and code up a set of loosely defined requirements, we spend a whole lot of time dealing with implementation details that along with the business logic will need to be redone if our interpretation of those requirements is somehow incorrect.

Ah, but isn't this what "refactoring" is all about? The answer to that question is "no". Refactoring is about cleaning up an implementation. It's about avoiding over-engineering by just writing the damn code, and then restructuring that code when it becomes unwieldy, rather than building elaborate frameworks before we truly understand what our implementation requires. "Requirements refactoring" is a euphemism for getting things wrong. When we need to "refactor code" to reflect a requirements "clarification", we are simply rewriting code that was written to do something nobody actually wanted.

If there were only some way to "code" requirements without having to deal with implementation details. Then, we could indeed use code as our medium for capturing precise requirements, without wasting a lot of time on unneeded implementation should those requirements prove inaccurate. Hmmm, if only there were some way to capture requirements as code without implementation details.... Oh, that's right. There is a way. And coincidentally, it's what we've been working on for the last six years here at GL.

Searching for love in all the wrong use cases

2008-11-06T20:42:00.000-07:00

Having established (ie, forcefully asserted) that it is generally Bad to engage in ritualistic development practices, but Good to adhere to a particular practice when it's well-suited to surmounting some challenge, let's consider the first snarling beast that leaps out to menace most young, impressionable development projects before they've even secured funding -- requirements definition.

Functional requirements are strange and mysterious things. Many programmers have never actually seen any, relying instead on rumor, innuendo or hearsay to guide their development objectives. For many, functional requirements are the only things keeping programming from being their dream vocation. Programming after all is, at its best, a beautifully artistic endeavor wherein we devise clever structures to manage the complexity of some problem, and transform chaos into order. Implementation "elegance" is regarded as one of the highest achievements we can attain as developers. As each new project begins, we think wistfully of the various forms of abstraction and all the intricate frameworks we have mastered over the years, and consider how, armed with the depth of our experience, this project will surely be different from our past ones. This time we'll build the thing Right, methodically assembling a great edifice from a series of simple building blocks. Each will be as cohesive as the nucleus of an atom, but as loosely coupled from one another as the grains of sand along the seashore. You kick off your shoes, and wander along the beach, smelling the salt air and thinking about that time under the boardwalk -- then, suddenly, a subject matter expert comes marching down the beach with several unruly business owners in tow, and all hell breaks loose.

Real-world requirements are seldom elegant. On the contrary, they tend to be maddeningly messy. For many applications, the hand-waving of some business stakeholder-guy can be sufficient for reaching an understanding of what needs to be built. But, alas, the majority of applications we build are far more "interesting" than these. Confronted by the terrifying unknown of some obscure workflow that has evolved over millions of years and countless mergers and acquisitions, many developers immediately fall back on their religious texts for guidance and comfort against the onrushing horror. They begin frantically writing user stories, drawing anatomically inaccurate stick figures, and in the dark of the night wondering why the gods of order and logic have forsaken them despite their best attempts to follow the good and righteous path documented and indexed in their most venerated religous text (second edition, totally updated with a new preface by the author).

Why must it be so? Why do business requirements, which on the surface seem so simple compared to the complexity of the software platforms we, as accomplished software developers, whip into submission almost every day, so often defy our best efforts to wrestle them to the ground though we pelt them with CRC cards and lock them in use cases? Is there no way to reduce the arbitrary tangle of business rules to some elegant expression of the problem that can return us to our idyllic afternoon at the seaside?

Grab a towel and some sunscreen and allow me to introduce you to Precision Requirements Modeling.

Too Much Religion

2008-11-04T16:44:00.000-07:00

Programmers are a religious sort, unquestioningly engaging in ritualistic practices pronounced as "good" by the industry's high priests, and likewise shunning those practices branded as blasphemous. Everyone knows for example that mixing business logic with user interface code will damn your soul to Hell for all of eternity. Likewise, engaging in waterfall development leads to nowhere but poverty and pestilence. And each time you use dependency injection, an angel gets his wings.

The religious tenets of our industry of course have mostly arisen from hard-learned lessons about what seems to generally work and what seems generally not to. Still, it is clearly the case that there is no single path to project nirvana. Every development initiative grapples with a unique variety of technical, political, and sociological challenges. Some projects even succeed. It is of course natural that people examine the practices used by the really successful ones, and strive to emulate them to maximize their own chances for salvation.

Successful software development, alas, cannot be achieved by blindly following the prescriptions laid down in some sacred textbook or blog posting. The sacred texts do indeed provide valuable techniques discovered only after the spilling of vast amounts of blood by our ancestors who came down out of their mainframes to spread out across the network, converting green screens to rich clients, and procedures to objects. So, let us embrace the tools and techniques, but let us also recognize that the reason software projects fail is that there is no "Way" that one can simply follow to guarantee success.

Software development projects present us with a series of challenges. Each of those challenges can generally be addressed with some established set of tools and techniques. When we complete a successful project we are typically tempted to look back on the steps we took along the way and announce to our colleagues, "Ah, yes, truly we have found The Way. Let The Way be written down so that it be made repeatably so, and we can look forward to nothing but happy days for ourselves, our managers, and our manager's managers." And yet we then see others try to follow The Way, only to find misery, chaos, and even death. Okay, maybe not death.

So are we condemned then to roll the dice each time we undertake a new development initiative? I don't think so. But the solution is not religion. The solution is in learning a few highly useful techniques, but rather than applying them ritualistically, we should instead learn to assess the particular challenges presented at a particular juncture of a particular project, and to select among and adapt the various techniques to the challenge at hand.

The Way does not exist. This is The Only Way.