http://money.cnn.com/magazines/fortune/fortune500/2012/full_list/ , slightly above the fifty line…
Yesterday’s build of a commercial application we are working on was reported as “with a bug”. An annoying dialog appeared here and there after a relatively long time of uninterruptible work of the application’s console. Need to say that reproduction of such bugs (“the console got tired and a dialog appears after an hour or more…”) is not a piece of cake. At least, no one wants to await the dialog in the debugger for an hour or two.
Now, the serious part of the product is being tested with PowerShell. Whereas the system part is being tested with C# (the ‘system team’ prefers writing code what I call ‘writing own bugs’), the UI including simple interactions like selecting users from a domain is done in PowerShell. The majority of the code exploits UIAutomation and TMX modules.
Below is the yesterday’s report (a TMX results report):
As can be seen, the average time of a test case is 8005/(2682-42)=3,03 seconds.
The fail/pass rate is 161/2682=6%.
Data is nothing if you have none to compare. Below is a report done a week ago (test cases number varies from run to run as the autotest developer and maintainer switches test cases on or off due to changes in the UI or for analysis and making amendments to the test suite):
The ‘happy’ run has 0,5% fail/pass rate (0.0% fail/pass rate is hardly achievable during the time the product’s UI is being changed).
The average time is better than in a normal case and counted as 4623/(2251-35)=2,08 seconds per test case, and there is still room for further improvements. Preferably, that fail-run increments were bound to 20%, not 50%.
Anyway, the red peak of 6% of the height of a percent chart seems appropriate. Isn’t it a success of using PowerShell in software testing?
Task: demonstrate how to write in text into a text box field.
Requirements: use UI Automation pattern or Win32.
Solution: the typical code is as follows:
Get-UIAWindow -Name $formTitle | ` Get-UIAEdit -AutomationId UsernameBox | ` Set-UIAEditText -Text "text";
Despite the name, the Set-UIAEditText cmdlet uses ValuePattern. This is because that Set-UIAEditText as well as any Set-UIA[ControlType]Text cmdlet is an alias of the Invoke-UIAValuePatternSet cmdlet.
the same story with the Set-UIATextBoxText cmdlet, which is the alias for Set-UIAEditText.
Not every time controls that are supposed to support ValuePattern really support it. This may depend on the developer’s code or on a control’s state. For example, a combo box supports ValuePattern if it has a field to write text in.
As promised, accessing such controls programmatically should raise the ModifiedChanged event.
Sometimes, the ValuePattern can be inaccessible. Due to many reasons, it happens, though rarely. Under these circumstances, you may use the Set-UIAControlText cmdlet, the pure Win32 text setter.
Get-UIAWindow -Name $formTitle | ` Get-UIAEdit -AutomationId UsernameBox | ` Set-UIAControlText -Text "text";
Notice, however, that it works slightly differently: it puts text before the text that is already in the text field. What means that you need first to clean it up, the control, before putting a new text in.
There’s been posted a request for more accurate method to get a window or control. Actually, getting a window by process name or window title may be considered by many as totally unreliable.
The application under test may be a page in the browser (several pages and browser instances are not rare these days), an MDI child, a frame. Simply, several instances of a GUI application can be running at the same time.
All these reasons led to adding new ways of getting windows.
Task: demonstrate how to get the window you’ll be using in tests by its process Id, window handle or process object.
Requirements: provide the reader with ready-to-run examples.
Solution: along with Get-UIAWindows -p (ProcessName) and -n (Name), the version 0.6.7 introduces new parameters for the Get-UIAWindow cmdlet.
- -ProcessId or -pid
Get-UIAWindow -pid (Get-Process -Name mmc).Id
- -Process or -p or -InputObject
Get-UIAWindow -p (Get-Process -Name mmc)
Earlier, many should do the following in order to start a process and get its handle:
Start-Process C:\Windows\system32\mmc.exe -Wait:$false Get-UIAWindow -pn mmc -Seconds 10
the code says to start a process and seek for its window for no more than ten seconds.
Now, we are definitely more sure about what process we are going to get. We simply pipeline it to a cmdlet:
Start-Process C:\Windows\System32\mmc.exe -PassThru | Get-UIAWindow -Seconds 10
The sample provided hooks the same window that we ran in it.
One more requested feature is to handle windows and control by its handle. Whence the handle can be taken?
One way is to get the property:
(Start-Process C:\Windows\System32\mmc.exe -PassThru | Get-UIAWindow -Seconds 10).Current.NativeWindowhandle (Start-Process C:\Windows\System32\mmc.exe -PassThru | Get-UIAWindow -Seconds 10 | Get-UIATree).Current.NativeWindowhandle
the other is to pipeline the handle out from the control:
PS C:\Users\apetrov1> Start-Process C:\Windows\System32\mmc.exe -PassThru | Get-UIAWindow -Seconds 10 | Read-UIAControlNativeWindowHandle Start-Process C:\Windows\System32\mmc.exe -PassThru | Get-UIAWindow -Seconds 10 | Get-UIATree | Read-UIAControlNativeWindowHandle
After you’ve got the handle, you are free to use it until it’s being disposed by the operating system:
Get-UIAControlFromHandle -Handle 67856 | Get-UIATreeItem -Name 'Console Root'
Task: demonstrate how to change the module settings in a bunch.
Task #2: provide an explanation what settings are.
Requirements: describe profiles and describe individual settings.
Solution: the UIAutomation module has a range of settigns. You can easily print them by running the following cmdlet:
The output provides you with a number of variables, some tell you something by their names, some don’t. Below are these settings:
C:\Users\Administrator> Show-UIAModuleSettings Timeout settings: [UIAutomation.Preferences]::Timeout = 20000 Error collection settings: [UIAutomation.Preferences]::MaximumErrorCount = 256 Common actions: [UIAutomation.Preferences]::OnErrorAction = [UIAutomation.Preferences]::OnSleepAction = [UIAutomation.Preferences]::OnSuccessDelay = Screenshot settings: [UIAutomation.Preferences]::OnErrorScreenShot = False [UIAutomation.Preferences]::ScreenShotFolder = C:\Users\Administrator\AppData\Local\Temp Log settings: [UIAutomation.Preferences]::Log = True [UIAutomation.Preferences]::LogPath = C:\Users\Administrator\AppData\Local\Temp\UIAutomation.log Debugging delays: [UIAutomation.Preferences]::OnClickDelay = 0 [UIAutomation.Preferences]::OnErrorDelay = 500 [UIAutomation.Preferences]::OnSleepDelay = 0 [UIAutomation.Preferences]::OnSuccessDelay = 500 Transcript settings [UIAutomation.Preferences]::TranscriptInterval = 200 Highlighter settings: [UIAutomation.Preferences]::Highlight = True [UIAutomation.Preferences]::HighlighterBorder = 3 [UIAutomation.Preferences]::HighlighterColor = Color [Red] C:\Users\Administrator>
What is what here? First of all, there is the need to mention also profiles (modes. Now, there is no decision on how to call them better :)).
Out of the box, you have the Presentation profile being set:
[UIAutomation.Mode]::Profile = [UIAutomation.Modes]::Presentation
This is colorful and not very fast. This is the face of the module: somebody downloads the module, tries to work with and shows to somebody else. The first thing the module should do well is to be accepted by the public. Why? Without the consumers, how should it progress?
Therefore, the first stage of the module life-cycle is being in the hands of newcomers (to the module). A tester, an IT manager or their boss(es), even the tester does through the newcomer stage.
Here comes the presentation mode: highlighting, half a second’s delays on every actions. Just to recline in the chair and watch the anime on the screen. Let’s explain these four settings:
[UIAutomation.Preferences]::OnClickDelay = 0
[UIAutomation.Preferences]::OnErrorDelay = 500
[UIAutomation.Preferences]::OnSleepDelay = 0
[UIAutomation.Preferences]::OnSuccessDelay = 500
If the cmdlet finished successfully (returned a control or called a pattern), it sleep for five hundred milliseconds. The time the spectators can use for exhaling and moving eyes.
If the cmdlet fails, it waits for 500 milliseconds, giving you the chance to stop the presentation or to continue if it’s not critical.
ONClickDelay is used in Win32 click cmdlets like Invoke-UIAControlClick and Invoke-UIAControlContextMenu.
OnSleepDelay is used in specific cases as if your host is overloaded and there is no reason to try to get an AutomationElement without a delay.
When you are debugging the script, the Debug mode is much more useful. It sleep a second after the success and five seconds after a failure.
When you are leaving the office and the script will work unattended, set the Normal mode, where there are no delays at all.
Moreover, all the settings, delays and others, you are allowed to change manually.
Homework: try to play with profiles (modes). Change individual settings. Observe, what are results. Change modes over any individual settings were done. Try change them in any combination.
Problem: the framework behaves strangely.
Requirements: reclaim the stability.
Solution: run your powershell console or IDE under domain admin credentials or through Run As Administrator. When working with something like Administrative tools (services.msc or compmgmt.msc, or ADUC), or regedit, or your company’s administrative tool, there might be a waterfall of errors, sometimes amazing. The answer is simple: use administrative credentials while working with administrative applications through inter-process communication.
Task: explain how to make your UI test script working in the VMware environment.
Requirements: provide scripters with recommendations.
Solution: sometimes you need to do a specific click. It might happen if a control does not support InvokePattern (thus, there’s no Invoke-UIA[ControlType]Cmdlet or there IS a cmdlet, but the control does not support the InvokePattern on its own). Usually it is not a problem: we have a Win32 click (Invoke-UIAControlClick) and cmdlets based on this click (Invoke-UIAControlContextMenu).
All of a sudden you may notice that surprisingly your click does not work. You are running the script on a VMware guest and the problem is that the cursor refuses positioning. In other words, the API function SetCursorPos does not work.
Theoretically, there might be used OpenInputDesktop, SetThreadDesktop and similar stuff. However, there is a much simplier way: just uninstall VMware Pointing device. With losing a bit of comfort, you’ll get your script working.
Task: start working with the script recorder.
Requirements: to be able to work with the recorder from the command line.
Solution: The script recorder is the cmdlet that was in the framework from almost its commencement. In fact, how to write the code? The scripter needs to have the following:
– a tool for recognizing AutomationID, ControlType and Name of controls that are under test
– a tool that can inform about supported patterns
– knowledge about cmdlets and parameters, some experience with the framework
– log problems
A tester needs significantly more:
– a way to get and store test results
– detailed log
– saving screenshots on any situation when it’s required
– custom parameters (the need to parametrize as much as possible to multiply the tests)
The script recorder helps in the needs of the scripter. The needs of a tester are resolved in the framework by different means. Issue the following command:
Start-UIARecorder -NoClassInformation -NoUI -NoScriptHeader -WriteCurrentPattern -Seconds 60
Now you can move the mouse cursor to the controls of your interest, performs actions like clicks and editing the text. The script recorder is far away of being finished, nevertheless its help may be very useful.
Remember that for now it records all it managed to record without separation from unnecessary windows and controls. Nevertheless, it might help at least in syntax.