The new elements’ object model is out. A short intro
Many claim that it’s not so easy to decide which pattern cmdlet to use with a particular control. For long time we were tied to what Microsoft’s UI Automation outputs to us. However, there is a way to increase productivity: extension methods. Yes, they are not visible in PowerShell by default, nonetheless I’d like to offer an object model that grew from nothing else but extension methods.
To start, let’s page through the presentation, with binaries of 0.8.7 alpha 3 or higher in your hands. Enjoy!
Desktop automation: towards code simplicity. Part 3
Not only in the Get-UIA[ControlType] cmdlet family were introduced improvements. Another veteran, the Get-UIAWindow cmdlet has been modernized.
This is a start place for our experiments:
# there's an application with two forms open Start-Process calc -PassThru | Get-UIAWindow | Get-UIAMenuItem help | Invoke-UIAMenuItemExpand | Get-UIAMenuItem -n about* | Invoke-UIAMenuItemClick;
Earlier, we had such options to get a child window:
# using the Get-UIAChildWindow cmdlet Get-UIAWindow -pn calc | Get-UIAChildWindow; # using recursive search for a control of window type (in practice, the same as using the Get-UIAChildWindow cmdlet) Get-UIAWindow -pn calc | Get-UIAControl -ControlType Window;
These samples worked well for us until there were three or more windows open (the main form, the child form and a message box, for example. Another case is the main form and two child forms) or the main form is so complex that consists of two or more forms (dockable parts or a window inside an MMC application).
In such cases, the search for a certain window led to complicated code, especially when forms had the same captions.
Time changed, let’s review new ways of getting a certain window:
# the topmost window (there are no changes with the previous behavior) Get-UIAWindow -pn calc; # all windows of the process(es): Get-UIAWindow -pn calc -Recurse; # all windows with the phrase 'calc' in caption Get-UIAWindow -pn calc -Name *calc*; # all windows with the word 'about' in caption (in fact, it's only the child window) Get-UIAWindow -pn calc -Name *about*; # all windows with ClassName like 'frame' (the main window has Classname == 'CalcFrame') Get-UIAWindow -pn calc -Class *frame*; # all windows with ClassName == '#32770' (the child window has such a standard class name) Get-UIAWindow -pn calc -Class '#32770';
As can be easily seen, one call of the Get-UIAWindow cmdlet could substitute an old-days function of getting the right window.
The only problem in recursive search for windows is the time it could take: if forms are full of controls, the search will relatively slow.
Desktop automation: towards code simplicity. Part 2
Just to drop a couple of words about parameterless search. How many words could be used describing parameterless search?
Let’s dive into samples:
# the classic parameterless search for a control Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick; Get-UIAText 123;
# here we search for a control that has handle Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick; Get-UIAText 123 -Win32;
# using wildcards in parameterless search (-Win32 only) Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick; Get-UIAText 12* -Win32;
All three samples return the same: two labels (texts). However, you should understand that
- the first sample (i.e., Get-UIAText 123) works via MS UI Automation. This means that there is no use of wildcards. The query is (ControlType.Text AND (Name == ‘123’ OR AutomationId == ‘123’ OR ClassName == ‘123’ OR ValuePattern.Value == ‘123’)).
- the second and the third samples work via Win32 API (SendMessage) and check for the value given every AutomationElement with handle of given type
The only disadvantage of this syntax is a habit to supply all names with asterisks:
# MS UI Automation does not like asterisks: Get-UIAText 12*; # use the -Win32 parameter if applicable Get-UIAText 12* -Win32; # or the exact name, automation id, class name or value of a control Get-UIAText 123;
In simple situation this syntax helps make your code even simpler:
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAMenuItem view | Invoke-UIAMenuItemExpand | Get-UIAMenuItem worksheets | Move-UIACursor -X 10 -Y 10 | Get-UIAMenuItem mortgage | Invoke-UIAMenuItemClick;
Desktop automation: towards code simplicity. Part 1
As you could notice, versions 0.8.3, 0.8.4 and 0.8.5 of the UIAutomation module took a few steps to be simpler and more friendly to the user. We carried out the following improvements to the module:
- added a Win32 click to the InvokePattern cmdlets (i.e., Invoke-UIAButtonClick, Invoke-UIAMenuItemClick, etc) in the case when pattern failed
- added the -Value parameter to Get-UIA[ControlType] cmdlets (i.e., Get-UIAEdit, etc)
- added the parameterless search like Get-UIAButton Next
- and we are working on an improvement to Set-UIAControlText and Set-UIAControlKeys cmdlets
Today’s post sheds light on the brand-new -Value parameter. Many controls support ValuePattern. The preceding link and this one give us the following definitions: “Represents a control that has an intrinsic value that does not span a range and can be represented as a string. This string may or may not be editable depending on the control and its settings.” “Allows clients to get or set a value on controls that do not support a range of values. For example, a date time picker.”
In practice, ValuePattern’s value is a string in a text field. Typically, it’s not wise to search for a text box by the value that is inside. It may be useful in certain situations (i.e., text boxes that are hard to recognize, for example, those that have auto-generated ids in AutomationId. And, in such situations, search for a control by a value in the field could help).
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [123] | Invoke-UIAButtonClick; Get-UIAText -Value 123;
A better example is Computer Management -> Disk Management: while names of drives’ squares say the number of device and partition, values contain descriptive information:
Get-UIAWindow -Name *comp*manag* | Get-UIACustom -Value *unallocated*;
WWhere else the -Value parameter can be used? Accordingly to the suggested controls to patterns mapping, ValuePattern is useful when automating links.
For example, a link in an event’s help page in Event viewer:
Get-UIAWindow -n *event*event* | Get-UIAHyperlink -Value *event*log*online*;
What’s more, link’s href is available even it’s invisible:
Get-UIAWindow -n *control*pattern*mapping* | Get-UIAHyperlink -Value '*#control_mapping_clients*' | Read-UIAControlName;
Finally, the best area of applicability of the -Value parameter is grids and other complex controls.
Daily automation: SendKeys tricks
Today’s comment is absolutely worth posting here!
How to send a special character via SendKeys – it may be not so easy as I thought.
First of all, how to do this task in an easy way (i.e., using ValuePattern). I created a simple Windows Forms application called ‘testeditbox’ with a label and a text box (the label is not used in the test, just a typical case text box + label).
We can write in and, after, clean it the following way:
Get-UIAWindow -n testeditbox | Get-UIAEdit | Set-UIAEditText texttexttext;
Get-UIAWindow -n testeditbox | Get-UIAEdit | Set-UIAEditText "";
Now, we are going to perform this without using the pattern. There are two ways to put text in: Set-UIAControlText and Set-UIAControlKeys.
The first method can’t send special characters (it’s just my fault, I need to add the possibility), it works through the SendMessage API call. It should work pretty well as it sends to a handle.
The second is .NET SendKeys.
The problem is that SendKeys sends only to the focused control. Well, I achieved the result this way:
Get-UIAWindow -n testeditbox | Get-UIAEdit | Set-UIAFocus; Get-UIAEdit | Set-UIAControlKeys "text2";
Get-UIAWindow -n testeditbox | Get-UIAEdit | Set-UIAFocus;
Get-UIAEdit | Set-UIAControlKeys "^(A){DELETE}";
Step by step guide:
1) get a control
2) set focus
3) get a control again (the Set-UIAControlKeys requires input), the control is foreground now
4) put keys to the control (Ctrl+A and Delete).
If your control of some other sort, you may need to do other tricks. Nonetheless, the idea behind is as I demonstrated.
Daily automation: getting selected radio button
This is truth, the phrase “only one who completely understood the topic can explain it to others” and “to put it so simply that even a five-year old guy would get it”. When I tried to write how to get an item that is selected, such as a radio button, I discovered that it’s not so easy as it should be. If a control containing items does not support SelectionPattern, there is a problem.
This fact led to creation of the Get-UIASelectedItem cmdlet.
To demonstrate how it works, let’s start calc.exe in the Programmer mode:
ipmo [path]\UIAutomation.dll Start-Process calc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name View | Invoke-UIAMenuItemExpand | Get-UIAMenuItem -Name Programmer | Invoke-UIAMenuItemClick;
The calc window in this mode looks like:
There are two sets of radio buttons. Unfortunately, containers of these controls do nothing, as they don’t support SelectionPattern. It would be excellent if the following code worked:
# this code does not work Start-Process calc -PassThru | Get-UIAWindow | Get-UIAImage -Name 'some name' | Get-UIAPaneSelection;
The problem is that none of two images supports SelectionPattern (and this is very atypical for an Image to support the most of the patterns).
This fact forces us to find out a solution using only those patterns that available to us. We know that radio buttons are likely to support SelectionItemPattern. Excellent! Let’s use this pattern.
We can group controls and get only those that are selected:
# this code does not work Get-UIAImage -AutomationId 1080 | Get-UIARadioButton | Get-UIASelectedItem;
Why does this code not work? The answer is simple: images are not containers for radio buttons. Even though two groups of radio buttons work simultaneously, i.e. there can be selected two radio buttons, one for each image, we can’t rely on images. All radio buttons, by the opinion of MS UI Automation, are under the Pane that includes the most of controls.
Get-UIAPane;
What should we do to get the selected radio button on the upper image? We can group radio button manually and put them as an input to the Get-UIASelectedItem cmdlet:
"hex","dec","oct","bin" | %{ Get-UIARadioButton -Name $_; } | Get-UIASelectedItem | Read-UIAControlName
Daily automation: Execution plan for UI Automation?
Have you tried to debug GUI or UI Automation scripts? It’s possible, of course, however this activity is often like making a mountain out of a molehill. Why?, may you ask. Because, because you need to run part of your scripts, or the whole suite, in the debugging mode, and this disturbance is, most often, from a single error.
It might be time-consuming. It’s not so pleasant: you need to interrupt your activity and do something another. Maybe, insert more Write-Host or Write-Verbose statements. Or extend the log. Or run the code inside command-line or GUI-based debugger.
Well, something simple you can now do directly from your script! A couple of months ago, the most of cmdlets (I’m about he UIAutomation module) could return only one object. Now, cmdlets return  all objects that match. Even absolutely all. This is a typical behavior. This is also a problem: imagine the following hypothetical code: you get a window, after that you want to get one or more panes, and, finally, buttons. Say, Close and Cancel.
Your code looks like:
ipmo [path]\UIAutomation.dll Get-UIAWindow -n "window name" | Get-UIAPane | Get-UIAButton -n C*;
Suddenly, you drove into an error. What? Error? I see the buttons, you think. The answer is simple: you’ve gotten two or three panes, but only one pane contains these buttons. You are given an error: the second and the third panes admitted that they lack buttons.
What would you do to resolve this? You need somehow learn which pane is yours. Here comes the Execution plan. What is it? I sure, many of us already know what an execution plan in MS SQL is. I was very excited about it, when, more than a decade ago, I’ve been achieving MCDBA. It’s how your queries will work and how long.
If you have never heard about it, several google results are worth being visited now: the search result.
Returning to our panes and buttons, we can now discuss what our plan consists of. UIAutomation 0.8.2. offers highlighter squares of ten colors for ‘generations’ of controls and numbers that allow you learn the order controls were output.
What is the thing a ‘generation’ of controls? I called so a wave of output. In our example, generation one is the window itself (it will be bordered and numbered 1). The second generation is panes (another color and numbers 2 in their upper-down corners). Finally, the third generation is buttons, also in another color.
Let’s run this code:
ipmo [path]\UIAutomation.dll Show-UIAExecutionPlan Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton; Get-UIAMenuItem; Get-UIAComboBox; Get-UIAEdit;
Here is how it looks like:
If we drag the window from under the ‘mesh’ of squares, what will be seen:
Number at the lower-right corners of squares follow the steps they have appeared:
- Get-UIAWindow
- Get-UIAButton
- Get-UIAMenuItem
- Get-UIAComboBox
- Get-UIAEdit
This may help in a complicated situation when you don’t know what path in the Automation tree your code walks.
Finally, there is a problem with PowerShell 2.0, or even powershell.exe -version 2. Thus, this feature is for PowerShell 3.0 running in the 3 mode.
Daily automation: dumping control’s genealogy
This is a new edition of the old post.
Note: this post is deprecated, please refer to my comment below.
One more way to find out where in the Automation tree a control resides is to use the Get-UIAControlAncestors cmdlet. We are told that TreeScope.Parent and TreeScope.Ancestors used as filter for search will call the functionality that is not yet implemented? However, the cmdlet does perform the search from the very bottom of the hierarchy to the topmost window. Let’s take Calculator and dump the control hierarchy:
Task:Â demonstrate using the search to the top cmdlet.
Requirements:Â perform search that consumes a part of control’s name and retrieve all the ancestors of the search result.
Solution:Â the following code prints out all the hierarchy in reverse order:
Set-StrictMode -Version Latest;
ipmo [path]\UIAutomation.dll;
# here we are searching for all the element whose names start with 'a'
Start-Process calc -PassThru | Get-UIAWindow | Search-UIAControl -Name A* | `
%{Write-Host "===========================================================";
Write-Host "@{Name='$($_.Current.Name)';
AutomaitonId='$($_.Current.AutomaitonId);
ControlType='$($_.Current.ControlType.ProgrammaticName)'}";
$_ | Get-UIAControlAncestors | `
%{Write-Host "@{Name='$($_.Current.Name)';
AutomaitonId='$($_.Current.AutomaitonId);
ControlType='$($_.Current.ControlType.ProgrammaticName)'}";}};
Daily automation: taking screenshots unleashed
By default, UIAutomation saves a screenshot if a cmdlet fails. This behavior is set by the default settings:
ipmo [path]\UIAutomation.dll [UIAutomation.Preferences]::OnErrorScreenShot [UIAutomation.Preferences]::ScreenShotFolder
The default folder for storing screenshots is user’s TEMP. Taking a screenshot can be initiated manually, below is how to do it:
ipmo [path]\UIAutomation.dll # saves the desktop Save-UIAScreenshot Save-UIAScreenshot -Description 123
The former line of code creates a file in the TEMP folder, for example: %USERPROFILE%\AppData\Local\Temp\20129261311418.bmp. The latter creates a file with more recognizable name: %USERPROFILE%\AppData\Local\Temp\20129261315161_123.bmp. Wanted better file names? Consider the following piece of code:
ipmo [path]\UIAutomation.dll # saves the desktop to a specified file Save-UIAScreenshot -Path c:\1 -Name 20120921_1.bmp
After we have discussed where we can save a screenshot, it’s time to discuss what can be saved. All the samples above saved the desktop – a reasonable choice if you don’t exactly know what the situation is when test fails. We can save a window, to the TEMP folder or to a folder of your choice:
ipmo [path]\UIAutomation.dll # saves a window Start-Process calc -PassThru | Get-UIAWindow | Save-UIAScreenshot # saves a window to the specified file Start-Process calc -PassThru | Get-UIAWindow | Save-UIAScreenshot -Path c:\1 -Name 20120921_2.bmp # throws an exception 'Save-UIAScreenshot : File 'c:\1\20120921_2.bmp' already exists' Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n 1 | Save-UIAScreenshot -Path c:\1 -Name 20120921_2.bmp
It’s not cmdlet’s concern if the file you specified already exists, so that it’s up to you to resolve such situations as shown in the code snipped above.
Finally, we can save any control, with or without a handle. The legend says that a control with handle is a window even if it’s a mere control. Times changes, controls without handles are now seen by UIAutomation as windows too. Here we get a button’s screenshot:
ipmo [path]\UIAutomation.dll # saves a button to the specified file Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n 1 | Save-UIAScreenshot -Path c:\1 -Name 20120921_3.bmp
The advantage of automation (as well as of UIAutomation) is that you can automate even already automated things. Ten screenshots at a time? It’s possible, below we produce screenshots for each of numeric buttons:
ipmo [path]\UIAutomation.dll
# generates a screenshot for each numeric button (ten screenshots)
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -SearchCriteria @{automationid='13*'} | Save-UIAScreenshot
Daily automation: more ways to perform a search
Initially, UIAutomation worked only with exact names, so that getting a control with several search conditions was an almost impossible thing:
ipmo [path]\UIAutomation.dll Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n 1;
When support of wildcards was introduced, search became useful as it was never before:
ipmo [path]\UIAutomation.dll Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n a*; Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n [0-9];
Nonetheless, even with wildcards, search capabilities were not so friendly as they could be. Thus, the search criteria were added to Get- cmdlets as they have already been in the Test-UIAControlState and Wait-UIAcontrolState cmdlets.
The code samples below return the same – numeric buttons in Calculator:
ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name [0-9];
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -AutomationId '13*';
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -SearchCriteria @{Name="[0-9]"};
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -SearchCriteria @{automationid="13*"};
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAControl -SearchCriteria @{Name="[0-9]"};
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAControl -SearchCriteria @{automationid="13*"};
Why do we need more ways for search? Sometimes, we need some kind of grouping of controls:
ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -SearchCriteria @{Name="[0-9]"},@{name="c*"};
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -SearchCriteria @{automationid="13*"},@{name="a*"};
In the sample, the first search returns numeric buttons plus the Clear, Clear entry and Close (a cross) buttons. The latter returns numeric buttons and the Add button.
There might also be searches like:
ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAControl -SearchCriteria @{isoffscreen="true"};
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAControl -SearchCriteria @{isenabled="false"};
What’s more, ‘search criteria’ strings (that are natural PowerShell hashtables) are way to store queries in a text file or a database instead of storing lines of code.
Software testing automation based mostly on PowerShell 