FlexMonkium 4.1.4 Released!

We are pleased to announce the newest version of FlexMonkium, Gorilla Logic's plugin that adds Flex recording and playback capabilities to the Selenium web testing framework. FlexMonkium 4.1.4, available for download from www.gorillalogic.com/flexmonkey.

FlexMonkium 4.1.4 includes some important enhancements, bug fixes and usability features including:

• Support for running FlexMonkey tests in Internet Explorer with Selenium-RC. Previous versions of FlexMonkium only supported Firefox.
• Bug fixes for the getFlexMonkeyValue command, which now reliably retrieves values from properties and table cells.
• Commands can now be made to wait for a component to appear before executing. For example, a command for clicking on a button will wait for the button to appear before attempting to click it.
• Ability to verify (or retrieve) values for property path expressions. For example, you can verify the number of rows in a DataGrid with the path expression "dataProvider.length".
• By default, all commands and verifies are now recorded as "WaitFor..." commands, which eliminates the need in most cases to slow or pause script execution to give components time to appear on the display.
• The XML recorded by Selenium-IDE for FlexMonkey commands has been simplified to make reading and editing easier.

FlexMonkium 4.1.4 is the successor to FlexMonkium 1.1. We changed the version numbering so that it syncs up with FlexMonkey version numbering (which we in turn changed last year to sync up with Flex version numbering).

To upgrade to FlexMokium 4.1.4, install the new console and download the new zip file. Upgrade the Firefox plugin by opening flexmonkium.xpi in Firefox. You will need to compile your app with the latest automation_monkey swc file. You will additionally need to use the new version of user-extensions.js with Selenium-RC.

Happy testing!

Great In-Depth Guide to FlexMonkey

Jon Rose has just published an extensive guide explaining the finer points of using FlexMonkey. You can check it out here!

New FoneMonkey for iOS Release!

We've just released the latest version of FoneMonkey, our free and open source functional testing tool for iOS applications on the iPhone, iPad, and iPod Touch.

You can find it at http://www.gorillalogic.com/fonemonkey.

Happy testing!

See FlexMonkey at Flex Camp Wall Street

I'll be presenting a session on FlexMonkey at Flex Camp Wall Street on November 15th in New York. Come on by to see the latest version of our open source Flex testing tool and let us know what you'd like to see it do in the future.

See FoneMonkey at 360|iDev

I'll be presenting a session on our open source FoneMonkey testing tool for iOS applications at next week's 360|iDev conference in Austin, TX.

If you're going to be at the conference, come on by to see how to get the most out of FoneMonkey, and let me know what you'd like to see in future releases.

FlexMonkium is here!

We are very excited today to announce the availability of FlexMonkium, a plugin that adds FlexMonkey recording and playback to Selenium, the popular, open source, web testing framework.

FlexMonkium provides for the testing of hybrid applications consisting of both Flex and HTML components by seamlessly interleaving the recording and playback of Flex and HTML interactions. Recorded tests can be output as JUnit-based scripts that can be extended with Java-based testing logic, and that can be run from build scripts and continuous integration processes.

You can download FlexMonkium from http://www.gorillalogic.com/flexmonkium.

FoneMonkey - A Closer Look (Clever Tester Managazine)

Recently I was interviewed by Clever Tester about FoneMonkey. You can check it out here.

FlexMonkey 4 and FlexMonkium (DZone)

I was recently interviewed by DZone about FlexMonkey 4 and FlexMonkium. You can check it out here.

Flex Developer's Journal: Functional testing of Flex RIA with FlexMonkey 4

I was recently interviewed by Yakov Fain, editor of Flex Developer's Journal. You can check it out here.

FlexMonkium: Get ready for the 119th element!

FlexMonkium, the Selenium Plugin that adds FlexMonkey recording and playback to Selenium-IDE and Selenium-RC, is undergoing final testing and packaging in preparation for its forthcoming release.

In this pair of videos smuggled from Gorilla Logic labs, deep beneath the surface of the Earth, we see FlexMonkium interleaving Flex with HTML recording and playback, and generated JUnit test scripts being run with Selenium-RC.

FlexMonkey 4 is here!

We just posted the new installer and accessories zip to FlexMonkey Project Home. Getting FlexMonkey to work with Flex 4 proved to be a bit more work than we'd anticipated but ultimately we got everything working and the result is a solid release that includes support for all Spark components.

Happily, we now return our attention back to FlexMonkium, our FlexMonkey/Selenium bridge, which is rapidly nearing it's first public release!

FlexMonkey 4 is coming!

We've just completed testing and anticipate releasing FlexMonkey for Flex 4 before the end of April!

This upcoming release, which we think we're going to call "FlexMonkey 4", works perfectly with Flex 4 and handles all of the new Spark components.

FoneMonkey 0.7.2 Released

Thanks to the continuing feedback from our rapidly growing community of FoneMonkey users, we've been able to identify and fix several issues and bring you FoneMonkey 0.7.2, available now at FoneMonkey Project Home.

This release fixes various bugs including problems with keyboard and scroll recording.

FoneMonkey 0.7.1 Released

Thanks to the intrepid souls who have taken the plunge into the first ever public release of FoneMonkey, we got some great feedback from 0.7 users and have just posted a new release, 0.7.1, with several important bug fixes and feature enhancements, notably:

Enhancements:

* WaitFor command - You can cause script execution to wait for views to be created or property values to be set
* "Popup Dialog" handling - You can now script UIAlertViews and UIAlertSheets.
* Better control event handling for UITextFields.

Bug fixes:

* OCUnit runner crash
* UINavigationBar rightItem playback


FoneMonkey 0.7.1 can be downloaded from FoneMonkey Project Home. We can understand if you considered 0.7 to be too early a release to try, but surely you'll feel differently about 0.7.1, which includes not one, but two, decimal points.

In all honesty, FoneMonkey, while not yet bug free, is quite stable and will definitely make your life better, at least in terms of iPhone testing.

Selenium Assertions and waitFors in FlexMonkium

In our previous video, we saw recording and playback of Flex interactions interleaved with browser interactions within a single Selenium script. Today's video shows how we can execute FlexMonkey Verify commands as assertions or waitFor predicates within a Selenium script.

The video shows a simply script being played back at slow speed. When we then try to play the same script back at fast speed, it fails because the script navigates to the page containing a Flex app, but then attempts to interact with it before the app has finished loading.

What we would like to do is have a way to tell Selenium to wait for some condition that signals the app has finished loading. One way to do this is to have Selenium wait for the appearance of a component within the application's window. We interactively create a FlexMonkey Verify command that tests whether the name field has a blank value. Notice that when we create this command, it is also "recorded" by the Selenium IDE as an assertion, assertFlexVerify.

This assertion will fail if it is executed before the swf has finished loading. However, we can ask Selenium to wait for the condition to become true by changing the assertFlexVerify command to be a waitForFlexVerify instead. Now, when we run the script, it waits for the swf to load before continuing, and then successfullly plays the rest of the script at high speed. We can of course use a similar technique to have a Selenium script wait for other Flex application events, such as data being returned from a query to be displayed in a table.



As you can see, we have all the major mechanisms necessary to very clean integration of FlexMonkey and Selenium. There is some trivial syntactic sugar we will likely sprinkle on what the video shows here, and we've still got some code clean up to do before we'll be ready to release, but we believe all the hard problems have been solved so it won't be much longer before FlexMonkium, the FlexMonkey Plug-In for Selenium, is publicly available!

FlexMonkium: The Movie

Once again, we bring you a video smuggled at great risk from Gorilla Logic Labs, deep beneath the surface of the Earth.

Last week, on this very blog, the world first learned of FlexMonkium, the FlexMonkey plug-in for Selenium. Now, we are pleased to present this exclusive video of FlexMonkium in action, allowing Selenium IDE to record both browser and Flex application interactions within a single Selenium script.

Although the recorded scenario is seemingly simple, it is actually quite wonderful. Selenium IDE begins recording on a plain html page, and continues recording as we navigate to a page containing a Flex application, and Flex application interactions are recorded to the script. Recording continues as we hit an html link ("Back") outside the Flex application in the browser page. For good measure, we continue recording as we navigate again to the page with the Flex app for further interaction, and then navigate back to the plain html page. All Flex and browser interactions have been recorded to a single Selenium script!

And of course, playback works as well.



Just a few more weeks and we'll be ready to release FlexMonkum! If you're interested in early access, please let us know.

We've released FoneMonkey, our record/play functional tester for iPhone

We officially launched the FoneMonkey open source project today at http://www.gorillalogic.com/fonemonkey.

If you've been following my recent posts, you know that FoneMonkey is the world's first record/playback functional testing tool for the iPhone. FoneMonkey records user interactions with the iPhone, and replays them while verifying that actual results match expected ones.

Key FoneMonkey features:

• High fidelity recording and playback of most iPhone gestures including touches, drags, and shakes.
• Integrated script editor and test runner.
• Recording and playback on both the simulator and actual iPhone device.
• Extensible framework can be customized to provide specialized command recording and playback for custom components.
• Simple Objective-API can be used to run FoneMonkey scripts using OCUnit.

With complete source, downloads, documentation, video tutorials, and a user forum available now at FoneMonkey Home, we are fully ready to monkey!

FlexMonkium: FlexMonkey/Selenium Bridge

Hey, all you Selenium people out there, get ready to monkey!

We've been getting a lot of requests about integrating FlexMonkey with Selenium to provide testing of "hybrid" test scenarios involving both Flex and non-Flex web apps. It is of course common to embed Flex applications within web pages that also contain non-Flex, HTML components, and the combined Flex and non-Flex pieces together produce a single user experience and workflow. Ideally, such hybrid user interface scenarios could be automated within a single test case. In fact, many times the functional dependencies between Flex and non-Flex components make it impossible to test one without the other.

While we've been hearing about many efforts to combine FlexMonkey with Selenium to provide for such hybrid testing, we're not aware of anything having yet been made publicly available, and having just completed development of FoneMonkey (which we're officially announcing tomorrow), and FlexMonkey 1.0GA (which we're releasing in about a week), we've had some time to take a look ourselves at how we might go about integrating Selenium with The Monkey.

After exploring various permutations, we hit upon a very robust approach through which we "slave" FlexMonkey recording and playback beneath Selenium. We have created a bridge, which we are provisionally calling FlexMonkium, that forwards FlexMonkey commands to Selenium during recording, and forwards FlexMonkey commands from Selenium to FlexMonkey during playback. Such FlexMonkey recording and playback is seamlessly interleaved with native Selenium recording and playback to easily produce hybrid testing scenarios. For example, in a single recording session you can click on HTML components and Flex components, even across multiple browser pages, with the resulting FlexMonkey and Selenium commands being recorded into a single Selenium script which (with the aid of the FlexMonkium bridge) can then be played back with any Selenium runner.

We are fairly confident we'll have something ready for early access in a matter of weeks! If you'd like to get your hands on FlexMonkium as soon as possible, contact us here.

FoneMonkey is In the Launch Tube

Although we're not officially launching for another week or so, the FoneMonkey project site is now live at http://www.gorillalogic.com/fonemonkey. We've published the source, binaries, and complete documentation. The issue management system and discussion board are also open for business.

iPhone developers of the world, get ready to monkey!

Drupal Page Editor Bookmarklet

We're in the process of putting gorillalogic.com under the Drupal content management system. I find Drupal's admin interface to be unbelievably annoying. What I really want to be able to do is navigate gorillalogic.com in my browser, and have a button I can hit to open the current page in the Drupal editor.

So, I created a bookmarklet for this purpose. Drag the link below to your browser's bookmark bar. When you're browsing a Drupal-managed site, click the bookmark to open the current page in the Drupal editor (assuming of course you have editing privilege for the page).

DrupEdit

Using FoneMonkey With OCUnit

FoneMonkey, the world's first functional testing automation tool for the iPhone, now provides direct support for running FoneMonkey scripts from OCUnit test suites.

You can download the latest version of FoneMonkey here.

Running Under OCUnit

OCUnit (also known as SenTestingKit) is a unit testing framework for Objective-C that's bundled with the XCode IDE. As a member of the xUnit family of testing frameworks, OCUnit is similar to other popular frameworks such as JUnit (for Java), and FlexUnit (for Adobe Flex).

Using OCUnit, you can combine FoneMonkey- and non-FoneMonkey-based tests into comprehensive test suites that can be invoked interactively, as well as from build scripts, and from continuous integration frameworks such as Hudson or Cruise Control. It is also possible to combine OCUnit tests with non-OCUnit (for example, JUnit) tests, and combine the output so as to provide considated test reporting across all front- and back-end components of your application.

Please note that to run FoneMonkey with OCUnit, you should not create a testing bundle. Instead, you use the FoneMonkeyOCUnit plug-in to launch your OCUnit tests cases, as described later in this section.

Adding OCUnit to Your Project

You add OCUnit to your project much in the same way you add any framework.

• Right click on the target that includes FoneMonkey, and select Get Info from the menu. The Target Info dialog will be displayed.
• Add a new entry to the Linked Libraries by clicking the add (+) button at the bottom of the dialog window. A selection dialog will open.
• Click the Add Other... button. A file selection dialog will open.
• Navigate to the /Developer/Library/Frameworks folder. Select SenTestingKit.framework.
• Click the Add button.

Adding the FoneMonkeyOCUnit Plug-In

The FoneMonkey distribution includes libFoneMonkeyOCUnit.a which is a static library that extends FoneMonkey with an OCUnit test launcher.

• To add the library to your project, right-click on your FoneMonkey testing target and select Get Info. The Target Info dialog will be displayed.

• On the Build Tab, scroll down to the Linking section. Add -lFoneMonkeyOCUnit to Other Linker Flags so that it now should look something like this:

  -ObjC -all_load -lFoneMonkey -lFoneMonkeyOCUnit 

Writing a Test Case

You write your test cases for FoneMonkey in the same way you write any OCUnit test. To create a test case, simply subclass SenTestCase and use STAssertions to test for expected results. For more information on OCUnit (SenTestingKit), see Apple's Documentation.

Using the FoneMonkey API, you can run FoneMonkey commands from within your test case, and you use OCUnit Let's look at an example:

---

#import 
#import 

#import "FoneMonkey.h"

@interface SampleTest : SenTestCase

- (void) testSample;

@end

@implementation SampleTest

- (void) testSample {
  NSString* lastResult = [[FoneMonkey sharedMonkey] playAndWait:@"Test1"];
  STAssertNil(lastResult, lastResult);
}

@end

---

You run a FoneMonkey script from a test case by calling runScript:(NSString*)script, where script is the name of a FoneMonkey script stored in your application's Documents directory. runScript: returns nil if the script runs successfully. For example, in the script above we call:

    NSString* lastResult = [[FoneMonkey sharedMonkey] runScript:@"Test1"];

If the script fails for any reason, runScript: returns the message generated from a failed Verify command, or other error message. You need to explicitly add an assertion to test for a non-nil result, and display the returned message if it's non-nil as follows:

STAssertNil(lastResult, lastResult);

You can of course do more in your test case than simply run a script and test its result. You can include addition programming logic and assertions, and run multiple scripts.

Using the FoneMonkey API, It is also possible to build scripts programatically at run time, rather than running a script previously saved.

Running OCUnit Tests

When you link your application with FoneMonkey and FoneMonkeyOCUnit, FoneMonkey will by default run all SenTestCase subclasses, and OCUnit output will be written to the console.

If you would like to exit your application after the OCUnit tests have run, set FM_ENABLE_AUTOEXIT the environment variable.

You can disable the running of OCUnit tests upon FoneMonkey start up by setting the FM_DISABLE_AUTOSTART environment variable. Omitting the -lFoneMonkeyOCUnit linker flag will also disable the running of any OCUnit tests at startup.

Using the FoneMonkey Console

Creating and Editing Scripts with the FoneMonkey Console

When you launch an application that has been linked with FoneMonkey (see Setup Guide), the FoneMonkey Console is displayed on top of the application’s window.



To begin recording, touch the Record button. The FoneMonkey console will hide and you can manually test your application. As you interact with it, FoneMonkey records FoneMonkey commands corresponding to each user interface gesture.

If you stop interacting with the application for longer than the timeout interval (by default, 2.5 seconds), the FoneMonkey console will reappear on top of your application window.
If you are not yet done recording, click any where over the application to hide the FoneMonkey console and resume recording.

Touching the Pause button hides the FoneMonkey window and lets you interact with your application without recording. The console will reappear after the timeout interval. Touch anywhere on your application to re-hide the console and continue without recording.
Touch the More button to view the list of recorded commands.



Commands are displayed in the Command List table. The Timeout slider can be used to set the timeout interval anywhere from 2 to 10 seconds. If the console is reappearing more quickly than you want it to while the console is recording or paused, make the timeout longer.

Touch the Play button to begin playback. The console hides and FoneMonkey executes each command in sequence, generating user interface actions corresponding to each command.
By default, FoneMonkey pauses .5 seconds between commands. Sometimes you may need FoneMonkey to wait additional time at certain points in a script. For example, a command might need to wait until an animation finishes running. Use Pause commands (see command reference) to insert pauses between commands during playback.

Editing Recorded Commands

To edit a recorded command, touch the Edit button on the row in the command list to be edited. The Command Editor will be displayed.



Using the command editor, you can change the Command, Component Class, Monkey ID, or any of the associated parameters. When you’re finished editing, touch the Done button.

Inserting and Deleting Commands

Commands can be inserted and deleted using the Insert and Delete toolbar buttons, which display the Insert or Delete buttons in each row of the displayed script.
Saving a Script

To save a script, touch the Save Script button. The Save Script dialog will be displayed.

Enter a name for the script and hit the Save button. Scripts are saved in the application's Documents directory.

Running a Saved Script

To run a saved script, touch the Open Script button. The Saved Script List will open displaying all saved scripts.


To open a script, touch its name in the list. The script will open in the console and can be played and edited.

Recording a New Script

To clear the currently opened script and begin creating a new one, click the Open Script button and then touch the New button displayed above the Saved Script List.

Deleting a Saved Script

To delete a saved script, touch the Open Script button and then click the Edit button displayed above the Saved Script List. You may then delete a saved script by touching the associated Delete Icon.

Working with FoneMonkey Command Scripts

FoneMonkey plays FoneMonkey commands. For each kind of user interface action there is a corresponding command. Some examples of commands are Touch, Scroll, and Shake. (See the FoneMonkey Command Reference for a complete list).

User interface actions are directed to components. For example, you Touch a UIButton, or Scroll a UITableView to Row 5. FlexMonkey commands identify the action to be performed, and the component to receive the action. Components are identified by a combination of their Object-C class name (or superclass name) and a string identifier called a monkeyID.

All commands are written as:

CommandName ComponentType "MonkeyID" Parameters, ...

where

CommandName identifies the command to be executed.
ComponentType is the Objective-C class name of the component to receive the command. If componentType is omitted, it defaults to UIView (ie, any component).
"MonkeyID" is a quoted string that uniquely identifies which instance of the component's class should receive the command. If there is only one instance of a particular type, monkeyID may be omitted.
Parameters, .... are zero or more command-specific parameter values in CSV format.

By overriding recording and playback methods, it's easy to add your own commands to FoneMonkey or customize existing commands. See the FoneMonkey Extension Guide for more information.

Some Command Examples

Touch the "Done" button:

Touch UIButton "Done"

Touch the PaintingView at coordinate (25, 75):

Touch PaintingView 25, 75

Enter text into the "FirstName" field:

InputText UITextField "FirstName" fred

Identifying Components by ClassName and MonkeyID

As mentioned above, commands correspond to user interface actions, and actions are directed to components. For example, a command might specify to Touch a UIButton. If there is only one UIButton on the screen, FoneMonkey knows which button the command is referring to. If there are multiple buttons, however, we must tell FoneMonkey which one to use. In addition to the component's Objective-C classname, and a unique identifier called a monkeyID that's generated by FoneMonkey extensions for various UIKit classes. For any command, you can specify just a className or a monkeyID, or you can specify both.

Component Identification Examples

If there are multiple buttons on the screen:

Touch UIButton "Done"

If there is just one button on the screen:

Touch UIButton

If threre is just one button on the screen with a monkeyID of "Done":

Understanding MonkeyID's

FoneMonkey's UIView extensions provide the default monkeyID for all components. By default, the monkeyID of a component is the value of its accessbilityLabel property, if one exists, its tag property if it's non-zero, and otherwise FoneMonkey generates a unique identifier as explained below. Many component types provide specialized monkeyID's. For example, UIButton returns its titleLabel.text as its monkeyID, UITextField returns its placeholder value.

See the FoneMonkey Command Reference for a description of the monkeyID's returned by each component type.

If a component provides no monkeyID, FoneMoney generates an identifier by assigning an ordinal to each instance of each class on the screen FoneMonkey generated monkeyID's are prefix with a #-sign.

Examples of Generated MonkeyID's

Touch the first button:

Touch UIButton #0


Touch the second button:

Touch UIButton #1

Validating Tests with the Verify Command

You insert Verify commands into your script to validate during playback that actual results match expected ones. The Verify command has the following syntax:

Verify ClassName "monkeyID" propertyPath, expectedValue

As with all FoneMonkey commands, you can specify either ClassName or monkeyID or both to identify the component.

propertyPath is any valid key path to a property.
expectedValue is the expected value of the property identified by the propertyPath.

Verify Command Examples

Check that the last name label is "Smith"

Verify UILable "Last Name" text, Smith

Check that the button says "Hello"

Verify UIButton titleLabel.text, Hello

If a verify command fails, a message is displayed in the FoneMonkey Command List.

Inserting Pauses into a Script

By default, FoneMonkey pauses .5 seconds between commands during playback. Sometimes you may need a longer delay between commands, for example if an animation needs to finish running before a component is displayed.

You can insert pause commands to allow additional time between the execution of commands. It has the following syntax:

Pause milliseconds

where milliseconds is the number of milliseconds to pause before executing the next command.

The Pause command ignores class name and monkey ID, if any are supplied.

Pause Example

Pause for 1 second:

Pause 1000

FoneMonkey: New Fat Library

Thanks to some helpful advice from Victor Costan about building fat libraries, the latest FoneMonkey library can be used to build applications for both the iPhone device and simulator. The fat library contains code for both arm7 (iPhone) and i386 (simulator) architectures, so you can reference the same library binary from either your simulator of device builds. Note that the linker will strip out unreferenced code so the convenience of the single library does not come at the expensive of a bloated application binary.

The download is available here.

FoneMonkey: The Update

This release of FoneMonkey, the world's first and only free and open source testing automation tool for the iPhone, fixes various bugs, adds support for additional Cocoa Touch components, and can be run on an actual iPhone device in addition to the simulator.

The latest FoneMonkey download and release instructions can be found here.

We expect to have the official FoneMonkey project site online later this month!