Windows Marketplace for Mobile: Certification Testing, Hopper, and Multiple Screen Support

Windows Marketplace for Mobile: Certification Testing, Hopper, and Multiple Screen Support:
You work very hard to develop, test, and market your Windows Mobile application. Failing Windows Marketplace for Mobile certification testing can be a frustrating experience, but it doesn’t have to be.  Failing certification costs a submission credit (or another $99 if you have exhausted your initial five credits), you have to correct and resubmit your application – likely not what you planned.  In this post, we’ll cover a few tips that should help you increase the first time pass rate and have your application published to the Marketplace sooner. 

TIP NUMBER ONE: Understand the Certification Requirements

As a Windows Marketplace for Mobile developer, you are notified via email when the Application Submission Requirements for Windows Marketplace for Mobile is updated. You can also check the Windows Marketplace for Mobile forum for important announcements including the availability of new requirements. You may be surprised the number of developers that attempt to pass certification testing without understanding the requirements. If you have any questions regarding the requirements, please post them in the very active Windows Marketplace for Mobile forum.

TIP NUMBER TWO: Run Hopper early and often

From the requirements document:

1.17. Hopper Test The application must complete two hours of Microsoft’s Hopper test without exhibiting unpredictable behavior, hanging or crashing.

The more you run Hopper, the more confidence you will have in passing this requirement. While the requirement calls for two hours, a good benchmark is the number of hours your app runs without problems. Use this as a development metric. There is no reason your application can’t run for 20 hours or more. A suggested practice is to let your application run overnight and check it in the morning when coming into work. Ideally, Hopper and your app will still be running!

Use the correct version of Hopper

There are separate versions of Hopper for Standard and Professional. Make sure you are using the correct one for the device / emulator you are testing against. Also Hopper has been updated to support Windows Mobile 6.5. The correct version of Hopper can be downloaded here.

Starting Hopper

Hopper is brain dead stupid. That is no knock against this stress testing tool. Hopper simply ‘taps’ and ‘types’ randomly into your application. It will stress your application like no user would in the real-world. It is great for exposing bugs that are not found during normal ad hoc testing. To ensure the device or your application is not hanging, it periodically pulls down the Start screen. It does not know anything about your application, in fact you have to artificially ‘force’ Hopper to tap and type into your application. This is done via a Focus Application. The focus application periodically (the default is every 10 seconds), forces your application into the foreground. As a developer, you need to customize the Focus App for your application, changing the string constant that references to your EXE name and path. Starting Hopper and the Focus app (you don’t need to explicitly start your application, it will be started by the Focus App) can be tricky, since Hopper immediately starts tapping and typing when started, making it difficult to navigate to start the Focus App. There are a couple of solutions to this:

1. Use RapiStart to remotely start the Focus App (this will implicitly start your application) and then Hopper in sequence. You can download RapiStart here: Windows Mobile Developer Power Toys.
2. Use the Managed Focus App. This tool is hugely convenient in that you specify all of the dependencies for Hopper in a config file and then just start the one application – no building necessary! It will launch your application and Hopper for you. I recommend this tool with a big caveat as some developers have had problems (Hopper failing after less than 30 minutes) with it. It seems to activate applications differently than does the native focus app. I would recommend using the native focus app first to establish a baseline. Then use the Managed Focus App. They should behave exactly the same. If not, stick with the native Focus App.
   

Hopper is stressing your app, therefore your app may crash

Note that Hopper is a stress tool and as such, your application may crash during a Hopper run. This is a good thing in that Hopper is doing its job stressing your app. It is a good practice to run your application under the Visual Studio debugger when running Hopper. Especially when you are early in the development cycle. This way when your app crashes, the debugger will stop exactly where the crash is occurring. You can also inspect the state of your app to help determine the cause of the crash.

Hopper failure: Start menu dead

One of the more common Hopper failures (log files are stored in \testlog\hopper.log) is ‘Start menu dead’. See here for a good explanation and possible causes of this failure: Hopper: Start Menu Dead!. The Hopper blog is also a good source for tips on running Hopper; however the content is skewed more towards OEMs.

TIP NUMBER THREE: Adapt Your App

From the requirements document:

1.13. Multiple Device Support
Developers must identify the required device functionality during the Windows Marketplace submission process. Applications must have the ability to run on multiple Windows Mobile device platforms, for example, multiple Standard or multiple Professional devices. Although not recommended, applications can support a limited number of Windows Mobile devices.

Screen Rotation

While not previously stated explicitly, this requirement also includes the proper handling of screen rotation. Just about every Windows Mobile Professional device now supports dynamic rotation. Typically this is triggered when sliding out an on-device keyboard or titling the device.
The Adapt Your App section in the Windows Mobile Developer center is a great resource for information on how to ensure your application properly adapts to different screen orientations, sizes, and DPIs.
When changing orientation the OS will send your application a size event. Make sure you are taking action on this event: OnSize (if a managed) or WM_SIZE (for native code). For example, you may need to reposition some of your user interface elements. For a great in-depth article on this topic, check out: Adaptable Apps For Windows Mobile.

Test, Test, and Test again

Take advantage of the plethora of emulators that are included with the SDKs / DTKs. The emulators support screen rotation (press the Calendar button) and come in multiple DPI and screen sizes. See here for a blog post on how to automate this process: PowerShell and Quick UI Check for WM Applications. Best case is to test on at least one device that ships in-market as some scenarios are difficult to test sufficiently within the emulator.

Need additional Help?

Hopefully these tips will help you avoid certification testing failures, if you need help with your specific test failure, use the Support link in the developer dashboard to create an email support request. Just be sure to include all identifying information regarding your application including the product id. The Marketplace certification team will respond within 48-business hours.
Thanks,
Mike