Tag Archives: UIAutomation module

Desktop automation: towards code simplicity. Part 3

Not only in the Get-UIA[ControlType] cmdlet family were introduced improvements. Another veteran, the Get-UIAWindow cmdlet has been modernized.

This is a start place for our experiments:

# there's an application with two forms open
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAMenuItem help | Invoke-UIAMenuItemExpand | Get-UIAMenuItem -n about* | Invoke-UIAMenuItemClick;

Earlier, we had such options to get a child window:

# using the Get-UIAChildWindow cmdlet
Get-UIAWindow -pn calc | Get-UIAChildWindow;
# using recursive search for a control of window type (in practice, the same as using the Get-UIAChildWindow cmdlet)
Get-UIAWindow -pn calc | Get-UIAControl -ControlType Window;

These samples worked well for us until there were three or more windows open (the main form, the child form and a message box, for example. Another case is the main form and two child forms) or the main form is so complex that consists of two or more forms (dockable parts or a window inside an MMC application).
In such cases, the search for a certain window led to complicated code, especially when forms had the same captions.
Time changed, let’s review new ways of getting a certain window:

# the topmost window (there are no changes with the previous behavior)
Get-UIAWindow -pn calc;

# all windows of the process(es):
Get-UIAWindow -pn calc -Recurse;

# all windows with the phrase 'calc' in caption
Get-UIAWindow -pn calc -Name *calc*;

# all windows with the word 'about' in caption (in fact, it's only the child window)
Get-UIAWindow -pn calc -Name *about*;

# all windows with ClassName like 'frame' (the main window has Classname == 'CalcFrame')
Get-UIAWindow -pn calc -Class *frame*;

# all windows with ClassName == '#32770' (the child window has such a standard class name)
Get-UIAWindow -pn calc -Class '#32770';

As can be easily seen, one call of the Get-UIAWindow cmdlet could substitute an old-days function of getting the right window.

The only problem in recursive search for windows is the time it could take: if forms are full of controls, the search will relatively slow.


Desktop automation: towards code simplicity. Part 2

Just to drop a couple of words about parameterless search. How many words could be used describing parameterless search?

Let’s dive into samples:

# the classic parameterless search for a control
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick;
Get-UIAText 123;


# here we search for a control that has handle
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick;
Get-UIAText 123 -Win32;


# using wildcards in parameterless search (-Win32 only)
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick;
Get-UIAText 12* -Win32;


All three samples return the same: two labels (texts). However, you should understand that

  • the first sample (i.e., Get-UIAText 123) works via MS UI Automation. This means that there is no use of wildcards. The query is (ControlType.Text AND (Name == ‘123’ OR AutomationId == ‘123’ OR ClassName == ‘123’ OR ValuePattern.Value == ‘123’)).
  • the second and the third samples work via Win32 API (SendMessage) and check for the value given every AutomationElement with handle of given type

The only disadvantage of this syntax is a habit to supply all names with asterisks:

# MS UI Automation does not like asterisks:
Get-UIAText 12*;
# use the -Win32 parameter if applicable
Get-UIAText 12* -Win32;
# or the exact name, automation id, class name or value of a control
Get-UIAText 123;

In simple situation this syntax helps make your code even simpler:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAMenuItem view | Invoke-UIAMenuItemExpand | Get-UIAMenuItem worksheets | Move-UIACursor -X 10 -Y 10 | Get-UIAMenuItem mortgage | Invoke-UIAMenuItemClick;

Java SWT apps official support

Even though we added support of Java SWT application months ago, in 0.8.0 Preview 5 or even earlier, we’ve never declared this fact. There’s a script within UIAutomation 0.8.5 packages that installs Java Access Bridge libraries by Oracle accordingly to this page: Installing Java Access Bridge.

All that you need to do is

  1. download the binaries (see inside the script)
  2. change a couple of variables to meet paths to Java on your host
  3. save and run the script As Administrator

Here are screenshots which abilities Oracle/Sun adds to our tools (Eclipse Indigo):

JavaSWTsupport001 JavaSWTsupport002 JavaSWTsupport003 JavaSWTsupport004 JavaSWTsupport005 JavaSWTsupport006 JavaSWTsupport007 JavaSWTsupport008









Desktop automation: towards code simplicity. Part 1

As you could notice, versions 0.8.3, 0.8.4 and 0.8.5 of the UIAutomation module took a few steps to be simpler and more friendly to the user. We carried out the following improvements to the module:

  • added a Win32 click to the InvokePattern cmdlets (i.e., Invoke-UIAButtonClick, Invoke-UIAMenuItemClick, etc) in the case when pattern failed
  • added the -Value parameter to Get-UIA[ControlType] cmdlets (i.e., Get-UIAEdit, etc)
  • added the parameterless search like Get-UIAButton Next
  • and we are working on an improvement to Set-UIAControlText and Set-UIAControlKeys cmdlets

Today’s post sheds light on the brand-new -Value parameter. Many controls support ValuePattern. The preceding link and this one give us the following definitions: “Represents a control that has an intrinsic value that does not span a range and can be represented as a string. This string may or may not be editable depending on the control and its settings.” “Allows clients to get or set a value on controls that do not support a range of values. For example, a date time picker.”

In practice, ValuePattern’s value is a string in a text field. Typically, it’s not wise to search for a text box by the value that is inside. It may be useful in certain situations (i.e., text boxes that are hard to recognize, for example, those that have auto-generated ids in AutomationId. And, in such situations, search for a control by a value in the field could help).

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick;
Get-UIAText -Value 123;


A better example is Computer Management -> Disk Management: while names of drives’ squares say the number of device and partition, values contain descriptive information:

Get-UIAWindow -Name *comp*manag* | Get-UIACustom -Value *unallocated*;


Where else the -Value parameter can be used? Accordingly to the suggested controls to patterns mapping, ValuePattern is useful when automating links.

For example, a link in an event’s help page in Event viewer:

Get-UIAWindow -n *event*event* | Get-UIAHyperlink -Value *event*log*online*;


What’s more, link’s href is available even it’s invisible:

Get-UIAWindow -n *control*pattern*mapping* | Get-UIAHyperlink -Value '*#control_mapping_clients*' | Read-UIAControlName;


Finally, the best area of applicability of the -Value parameter is grids and other complex controls.

UIAutomation and PowerBuilder 12.5

Many years ago, in 1999-2000, I used to use PowerBuilder. At first, it was 5.2 (a pretty buggy and unstable version as I remember), later I used 7.0. Even in these times, PowerBuilder has been collapsing. Delphi, Visual Basic, Crystal Reports, many newcomers struggled for that tasteful market, visual access to databases.

After, I tried 8.0. These times I thought that, as the majority of successful and lucrative start-ups, PowerBuilder was bought for the purpose to obtain its customers only, not the application itself.

I almost forgot this fun application, with the running man for start, pebbles for source code (sometimes source files behaved strangely. Joke? Nope, source files are not mere text files here. They are specific files for numerous platforms, therefore they could be broken, huh). DataWindows, source code editors with pink band for inherited code, etc, etc, … The old good app.

Recently, I saw a question in a LinkedIn group: “Any recommendations on a UI Automation tool, preferably open source or free, that works with PowerBuilder 12.0 and/or 12.5”.

I was surprised by the fact that somebody still needs testing against PowerBuilder, decided that SAP gave it the second life and downloaded 12.5.

My first impression was that PowerBuilder shipped in two editions: .NET and classic. I glanced at the .NET edition, got that it produced WPF applications and lost the interest.

The classic edition, on the opposite, took me. With nostalgia and disgust at all these fun but unfriendly menu and toolbars, I started testing all that I found. I discovered that I almost completely forgot how to build a window (the start-up script), attach a menu to it, attach a data window to it, etc, so that I turned to tutorials and samples. I could not run the tutorial because I forgot the password to the sample SQL Anywhere DB, and I opened samples.

1. Advanced GUI

Having started the sample,



this app appeared.


To my shock, it was too advanced: no accessibility is provided. While I could work by names with the tutorial app, Sybase left only AutomaitonId and ClasName for testers. Awful!

To open the Help/About window, testers need to run the following abracadabra:

Get-UIAWindow -Name advanced*gui* | Get-UIAMenuItem -AutomationId 'item 2' | Invoke-UIAMenuItemExpand | Get-UIAMenuItem -AutomationId 'item 10002' | Invoke-UIAMenuItemClick;

The old-school tab control and toolbars are Panes.
The grid is accessible, but, but…
This code gets the topmost field in the grid and turns John into Johnny:

Get-UIAWindow -Name advanced*gui* | Get-UIAText -Name first_name_t | Invoke-UIAControlClick; Get-UIAControlFromPoint | Set-UIAControlText -Text Johnny;


The rest of grid fields share the same name, what is even worse.

The second tab is better: names of control are available to testers.


Get-UIARadioButton -Name Horizontal | Invoke-UIARadioButtonSelectItem -ItemName Horizontal;


Other three tabs were not better: the most of the controls are Panes.
I checked other samples, some could be automated, some are problematical.

0.8.0 Preview 5 is available to download

UIAutomation 0.8.0 Preview 5 has been uploaded to

http://uiautomation.codeplex.com/releases/view/93646 .


Even though Preview 4 was to some extent working, we fixed several bugs that we allowed to pass to Preview 4.

  • if no control was found, the Win32 API (FindWindowEx) returned the full collection of controls with a handle – fixed
  • in a rare case when UIAutomation couldn’t find a window, the search was additionally performed by calling FindWindow. This ability has been broken in 0.8.0P4 – fixed
  • output of some cmdlets (used rarely) was broken (i.e., a hashtable was outputted as a collection of items) – fixed
  • the logical bug (that was explained in the previous post) with the turbo timeout auto-switching only on a ‘no window’ error was fixed by adding the -IsCritical parameter to Get-UIA[ControlType] and Wait- cmdlets

Now, UIAutomation cmdlets with new output pass tests for UIAutomation 0.8.0P3 with the old, restricted output.

0.8.0 Preview 4 is available to download

UIAutomation 0.8.0 Preview 4 has been uploaded to http://uiautomation.codeplex.com/releases/view/93118 and now you can try new output.

Except a couple of issues in the functionality that is not widely used, all our unit tests are green. Nonetheless, we’d recommend you to play with this release with some care: there might be combinations of code where new cmdlets may behave slightly differently.

For example,

Get-UIAWindow ... | Get-UIAButton ... | %{ $_ | Read-UIAControlName ... }

with this release can be written as

Get-UIAWindow ... | Get-UIAButton ... | Read-UIAControlName ...


0.8.0 Preview 4: no more restrictions on pipeline

In response to a request on the project forum, UIAutomation now supports outputting all the items that engine returns. How did it work earlier? Get-UIAWindow returned only one window, so did Get-UIA[ControlType] cmdlets.

How the most of traditional cmdlets work? Get-Process returns all the processes that match or simply all processes in the system. Get-ChildItem returns all the file system items in the current folder or wherever the -Path parameter said, if no filter supplied. Even Selenium cmdlets return all web elements that match a condition.

Thus, to meet standard practices, UIAutomation now returns all what match. Get-UIAWindow returns all ‘Calculator’ windows if the command a user issued was

Get-UIAWindow -Name calc*

Note that the [UIAutomation.CurrentData]::CurrentWindow variable (i.e. the current window is the last window object in the pipeline).
Get-UIAButton returns all buttons of the window if the least restrictive wildcard is given:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name *

Pattern cmdlets (Invoke-, Get- and Set-) and Read- cmdlets accept such output. Even Get-UIA[ControlType] -Win32 parameter returns not the only object as earlier but all that match. Note that -Win32 does not understand types of controls and returns all whose names match the condition:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name * -Win32 | Read-UIAControlName

The sample outputs names of all controls with handles.

Finally, if some of you readers wanted to click all the buttons at a time (I wanted), here is the script:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name * | ?{ ($_ | Read-UIAControlName) -ne 'Close' } | Invoke-UIAButtonClick

Test Case Management: generating test results on the fly

The truth is that almost nobody writes test cases these days by hands. Okay, for big and serious, and requiring mind efforts things like scenarios, testers write. PowerShell frameworks are all about automation and demand as little manual work as possible. Like other GUI and Web testing tools, PowerShell frameworks generate test results by watching the code execution. Let’s go to the samples:

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

What did we expect from this code? We set the EveryCmdletAsTestResult setting on, started an instance of Firefox, navigated to Google and submitted a query. As we saw no errors, we want to get our passed test results. How to do that?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is as follows:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q
Status : PASSED

Name : Set-SeWebElementKeys Cheese
Status : PASSED

Name : Submit-SeWebElement;
Status : PASSED

Name :
Every cmdlet reported itself as Passed, and five results display this. The sixth result is a pre-generated test result that will be used in the immediately next cmdlet call.

This is too ideal, now we change our code to obtain real-life results. We are seeking for one of Google 2.0 controls, namely ‘q2’ (Marissa Mayer’s gone, hasn’t she?).

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q2 | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

The code failed (controls have not been renamed yet), where are our results?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is below:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name :

The third and the fourth results have different exceptions in their Descriptions, though it’s a place where module’s code review needed…

Now, let’s see how time was consumed:

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status,TimeSpent

As can be seen, there is a default 500 milliseconds’ delay after starting the browser, 2 seconds were spent on navigation to the google page, and 5 seconds (the full time of [SePSX.Preferences]::Timeout) were spent on attempts to get the control of our interest.

In practice, testers are often interested in Failed results, much more often that in the list of Passed. 🙂 How to obtain such a list? First of all, we need to import the TMX module. Selenium and UIAutomation modules use TMX indirectly, as a library, but the advanced functionality is available as cmdlets:

ipmo [path]\TMX.dll;
Search-TMXTestResult -FilterFailed | FL Name,LineNumber,Code,Details

If the time of every test result is stored, what benefits do we have? Consider using the following query:

Search-TMXTestResult -OrderByTimeSpent -Descending -FilterFailed | FL Name,TimeSpent,Code

This lists Failed test results and time consumed in the descending mode (for what kind of user the contemporary versions of MS Excel have been written? Maybe, I’m wrong here? Okay, I’ll rewrite). This command lists Failed test results from bigger time spent to smaller.

Test Case Management: filtering and sorting the results

Let’s start from yesterday’s script. I added the code that never would work: getting a non-existing window and finding the “10” button on Calculator. Here is the script:

ipmo [path]\TMX.dll
ipmo [path]\UIAutomation.dll
Start-Process calc
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n add | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton | Set-UIAFocus | Set-UIAControlKeys -Text "1{+}1{=}"

Get-UIAWindow -n "non-existing window" -Seconds 2
Get-UIAWindow -n calc* | Get-UIAButton -n 10 | Invoke-UIAButtonClick;

Stop-Process -Name calc

Export-TMXTestResults -As HTML -Path c:\1\calc_test_results_generated.htm

The three lines copy-pasted from yesterday’s post do exactly what they did yesterday: generate thirteen test results. What do the lines I added? The first line tries to get the window that’s rarely seen in the real life (you may compile one, though). The next line searches for the 10 button (that Microsoft declined to add to the tool, with very oblique explanation).

Okay, what should we expect? The instruction ‘Get-UIAWindow -n “non-existing window” -Seconds 2′ fails after two seconds’ time, so does ‘Get-UIAWindow -n calc* | Get-UIAButton -n 10’.

What does it mean for the whole test? These test cases (i.e., commands) are considered as most time-consuming (several queries to the Automaiton tree until the timeout ends this) and, no doubts, failed.

All the test results that failed can be pulled out with the following code:

Search-TMXTestResult -FilterFailed

The worst test result, considering as the worst a test result with the greatest time spent, can be scooped up with the following piece of code;

Search-TMXTestResult -FilterFailed -OrderByTimeSpent -Descending

Test results that are generated automatically have names based on the code. This leads us to the opportunity to filter test results by a code snippet or a keyword:

Search-TMXTestResult -FilterNameContains Button


Search-TMXTestResult -FilterNameContains 10

All that described are going to be available today as part of 0.8.0. Preview 1.

%d bloggers like this: