Selenium Assertions and waitFors in FlexMonkium

In our previous video, we saw recording and playback of Flex interactions interleaved with browser interactions within a single Selenium script. Today's video shows how we can execute FlexMonkey Verify commands as assertions or waitFor predicates within a Selenium script.

The video shows a simply script being played back at slow speed. When we then try to play the same script back at fast speed, it fails because the script navigates to the page containing a Flex app, but then attempts to interact with it before the app has finished loading.

What we would like to do is have a way to tell Selenium to wait for some condition that signals the app has finished loading. One way to do this is to have Selenium wait for the appearance of a component within the application's window. We interactively create a FlexMonkey Verify command that tests whether the name field has a blank value. Notice that when we create this command, it is also "recorded" by the Selenium IDE as an assertion, assertFlexVerify.

This assertion will fail if it is executed before the swf has finished loading. However, we can ask Selenium to wait for the condition to become true by changing the assertFlexVerify command to be a waitForFlexVerify instead. Now, when we run the script, it waits for the swf to load before continuing, and then successfullly plays the rest of the script at high speed. We can of course use a similar technique to have a Selenium script wait for other Flex application events, such as data being returned from a query to be displayed in a table.



As you can see, we have all the major mechanisms necessary to very clean integration of FlexMonkey and Selenium. There is some trivial syntactic sugar we will likely sprinkle on what the video shows here, and we've still got some code clean up to do before we'll be ready to release, but we believe all the hard problems have been solved so it won't be much longer before FlexMonkium, the FlexMonkey Plug-In for Selenium, is publicly available!

FlexMonkium: The Movie

Once again, we bring you a video smuggled at great risk from Gorilla Logic Labs, deep beneath the surface of the Earth.

Last week, on this very blog, the world first learned of FlexMonkium, the FlexMonkey plug-in for Selenium. Now, we are pleased to present this exclusive video of FlexMonkium in action, allowing Selenium IDE to record both browser and Flex application interactions within a single Selenium script.

Although the recorded scenario is seemingly simple, it is actually quite wonderful. Selenium IDE begins recording on a plain html page, and continues recording as we navigate to a page containing a Flex application, and Flex application interactions are recorded to the script. Recording continues as we hit an html link ("Back") outside the Flex application in the browser page. For good measure, we continue recording as we navigate again to the page with the Flex app for further interaction, and then navigate back to the plain html page. All Flex and browser interactions have been recorded to a single Selenium script!

And of course, playback works as well.



Just a few more weeks and we'll be ready to release FlexMonkum! If you're interested in early access, please let us know.

We've released FoneMonkey, our record/play functional tester for iPhone

We officially launched the FoneMonkey open source project today at http://www.gorillalogic.com/fonemonkey.

If you've been following my recent posts, you know that FoneMonkey is the world's first record/playback functional testing tool for the iPhone. FoneMonkey records user interactions with the iPhone, and replays them while verifying that actual results match expected ones.

Key FoneMonkey features:

• High fidelity recording and playback of most iPhone gestures including touches, drags, and shakes.
• Integrated script editor and test runner.
• Recording and playback on both the simulator and actual iPhone device.
• Extensible framework can be customized to provide specialized command recording and playback for custom components.
• Simple Objective-API can be used to run FoneMonkey scripts using OCUnit.

With complete source, downloads, documentation, video tutorials, and a user forum available now at FoneMonkey Home, we are fully ready to monkey!

FlexMonkium: FlexMonkey/Selenium Bridge

Hey, all you Selenium people out there, get ready to monkey!

We've been getting a lot of requests about integrating FlexMonkey with Selenium to provide testing of "hybrid" test scenarios involving both Flex and non-Flex web apps. It is of course common to embed Flex applications within web pages that also contain non-Flex, HTML components, and the combined Flex and non-Flex pieces together produce a single user experience and workflow. Ideally, such hybrid user interface scenarios could be automated within a single test case. In fact, many times the functional dependencies between Flex and non-Flex components make it impossible to test one without the other.

While we've been hearing about many efforts to combine FlexMonkey with Selenium to provide for such hybrid testing, we're not aware of anything having yet been made publicly available, and having just completed development of FoneMonkey (which we're officially announcing tomorrow), and FlexMonkey 1.0GA (which we're releasing in about a week), we've had some time to take a look ourselves at how we might go about integrating Selenium with The Monkey.

After exploring various permutations, we hit upon a very robust approach through which we "slave" FlexMonkey recording and playback beneath Selenium. We have created a bridge, which we are provisionally calling FlexMonkium, that forwards FlexMonkey commands to Selenium during recording, and forwards FlexMonkey commands from Selenium to FlexMonkey during playback. Such FlexMonkey recording and playback is seamlessly interleaved with native Selenium recording and playback to easily produce hybrid testing scenarios. For example, in a single recording session you can click on HTML components and Flex components, even across multiple browser pages, with the resulting FlexMonkey and Selenium commands being recorded into a single Selenium script which (with the aid of the FlexMonkium bridge) can then be played back with any Selenium runner.

We are fairly confident we'll have something ready for early access in a matter of weeks! If you'd like to get your hands on FlexMonkium as soon as possible, contact us here.

FoneMonkey is In the Launch Tube

Although we're not officially launching for another week or so, the FoneMonkey project site is now live at http://www.gorillalogic.com/fonemonkey. We've published the source, binaries, and complete documentation. The issue management system and discussion board are also open for business.

iPhone developers of the world, get ready to monkey!

Drupal Page Editor Bookmarklet

We're in the process of putting gorillalogic.com under the Drupal content management system. I find Drupal's admin interface to be unbelievably annoying. What I really want to be able to do is navigate gorillalogic.com in my browser, and have a button I can hit to open the current page in the Drupal editor.

So, I created a bookmarklet for this purpose. Drag the link below to your browser's bookmark bar. When you're browsing a Drupal-managed site, click the bookmark to open the current page in the Drupal editor (assuming of course you have editing privilege for the page).

DrupEdit

Using FoneMonkey With OCUnit

FoneMonkey, the world's first functional testing automation tool for the iPhone, now provides direct support for running FoneMonkey scripts from OCUnit test suites.

You can download the latest version of FoneMonkey here.

Running Under OCUnit

OCUnit (also known as SenTestingKit) is a unit testing framework for Objective-C that's bundled with the XCode IDE. As a member of the xUnit family of testing frameworks, OCUnit is similar to other popular frameworks such as JUnit (for Java), and FlexUnit (for Adobe Flex).

Using OCUnit, you can combine FoneMonkey- and non-FoneMonkey-based tests into comprehensive test suites that can be invoked interactively, as well as from build scripts, and from continuous integration frameworks such as Hudson or Cruise Control. It is also possible to combine OCUnit tests with non-OCUnit (for example, JUnit) tests, and combine the output so as to provide considated test reporting across all front- and back-end components of your application.

Please note that to run FoneMonkey with OCUnit, you should not create a testing bundle. Instead, you use the FoneMonkeyOCUnit plug-in to launch your OCUnit tests cases, as described later in this section.

Adding OCUnit to Your Project

You add OCUnit to your project much in the same way you add any framework.

• Right click on the target that includes FoneMonkey, and select Get Info from the menu. The Target Info dialog will be displayed.
• Add a new entry to the Linked Libraries by clicking the add (+) button at the bottom of the dialog window. A selection dialog will open.
• Click the Add Other... button. A file selection dialog will open.
• Navigate to the /Developer/Library/Frameworks folder. Select SenTestingKit.framework.
• Click the Add button.

Adding the FoneMonkeyOCUnit Plug-In

The FoneMonkey distribution includes libFoneMonkeyOCUnit.a which is a static library that extends FoneMonkey with an OCUnit test launcher.

• To add the library to your project, right-click on your FoneMonkey testing target and select Get Info. The Target Info dialog will be displayed.

• On the Build Tab, scroll down to the Linking section. Add -lFoneMonkeyOCUnit to Other Linker Flags so that it now should look something like this:

  -ObjC -all_load -lFoneMonkey -lFoneMonkeyOCUnit 

Writing a Test Case

You write your test cases for FoneMonkey in the same way you write any OCUnit test. To create a test case, simply subclass SenTestCase and use STAssertions to test for expected results. For more information on OCUnit (SenTestingKit), see Apple's Documentation.

Using the FoneMonkey API, you can run FoneMonkey commands from within your test case, and you use OCUnit Let's look at an example:

---

#import 
#import 

#import "FoneMonkey.h"

@interface SampleTest : SenTestCase

- (void) testSample;

@end

@implementation SampleTest

- (void) testSample {
  NSString* lastResult = [[FoneMonkey sharedMonkey] playAndWait:@"Test1"];
  STAssertNil(lastResult, lastResult);
}

@end

---

You run a FoneMonkey script from a test case by calling runScript:(NSString*)script, where script is the name of a FoneMonkey script stored in your application's Documents directory. runScript: returns nil if the script runs successfully. For example, in the script above we call:

    NSString* lastResult = [[FoneMonkey sharedMonkey] runScript:@"Test1"];

If the script fails for any reason, runScript: returns the message generated from a failed Verify command, or other error message. You need to explicitly add an assertion to test for a non-nil result, and display the returned message if it's non-nil as follows:

STAssertNil(lastResult, lastResult);

You can of course do more in your test case than simply run a script and test its result. You can include addition programming logic and assertions, and run multiple scripts.

Using the FoneMonkey API, It is also possible to build scripts programatically at run time, rather than running a script previously saved.

Running OCUnit Tests

When you link your application with FoneMonkey and FoneMonkeyOCUnit, FoneMonkey will by default run all SenTestCase subclasses, and OCUnit output will be written to the console.

If you would like to exit your application after the OCUnit tests have run, set FM_ENABLE_AUTOEXIT the environment variable.

You can disable the running of OCUnit tests upon FoneMonkey start up by setting the FM_DISABLE_AUTOSTART environment variable. Omitting the -lFoneMonkeyOCUnit linker flag will also disable the running of any OCUnit tests at startup.