Tag Archives: no fragility

Daily automation: eliminating the fragility of tests. Part 3


Many of us faced the situation when a window is gone and a testing tool hits the air for long time. The testing tool tries to get a control, a next control, and so on. Every attempt takes time.

UIAutomation had the remedy for the problem for relatively long time. This feature has never been widely known, so that it’s time to tell you what it is and how it works.

Task: minimize the time that UIAutomation spends on controls of a window that failed to appear.

Requirements: demonstrate how to use the technique with default settings.

Solution: you don’t need to do anything to benefit from this technique in case if a window of your interest is gone:

ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow;
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout
Get-UIAWindow -Name calc1 # non-existing window
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout
Get-UIAWindow -Name calc* # this call get a window
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout

After we ran this sample, we have the following output:
5000 – the default timeout
2000 – the AfterFailTurboTimeout
2000 – the timeout equals to AfterFailTurboTimeout. This is because calc1 has not been caught and the default timeout was considered as too long to wait for nothing
2000 – the AfterFailTurboTimeout
5000 – the default timeout was restored after the test have got a window
2000 – the AfterFailTurboTimeout
Note: this funcitonality has been broken in 0.8.0 Preview 4 (more accurately, 0.8.0P4 was released despite the fact that several unit tests have not been fixed. Just because this functionality is not widely known and never used explicitly).

Internally, UIAutomation checks that [UIAutomation.CurrentData]::CurrentWindow is $null and [UIAutomation.CurrentData]::LastResult also is $null.

Well, what about controls? Imagine the case you have a window and a child window or a control failed to appear.

This problem is not so obvious as the ‘no such window’ problem. ‘One control is gone’ is a typical case during the development cycle. Nonetheless, some controls are vital for the whole test. For this, a new parameter is here: -IsCritical. Let’s see how it works:

ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 10
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 10 -IsCritical
[UIAutomation.Preferences]::Timeout
[UIAutomation.CurrentData]::LastResult
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1 -IsCritical
[UIAutomation.Preferences]::Timeout

Below is the output:
5000 – the default timeout
5000 – the default timeout as the -IsCritical parameter has not been used
2000 – the turbo timeout (thanks to the -IsCritical parameter)
5000 – the default timeout (after the first control that has been caught successfully)
5000 – IsCritical does nothing if there wasn’t a problem

The general rule is to use the -IsCritical parameter for child windows, default buttons and controls like combo boxes and radio buttons if their state blocks you from going forward.

Daily automation: eliminating the fragility of tests. Part 2


The logically next step, after we started searching for a control using a wildcard, is to get a window by part of its name.

Task: use UIAutomation to get a window by a fragment of its name.

Requirements: demonstrate how to use the technique.

Solution: code that is needed is as simple as it can be:

Set-StrictMode -Version Latest;
ipmo [path]\UIAutomation.dll;
Start-Process calc; Get-UIAWindow -Name *alc*r

After we started the calc process, all the lines of code below will work:

Set-StrictMode -Version Latest;
ipmo [path]\UIAutomation.dll;
Start-Process calc;
Get-UIAWindow -Name *alc*r
Get-UIAWindow -Name Calc*
Get-UIAWindow -Name CA?culator
Get-UIAWindow -pn calc
Get-UIAWindow -pid (Start-Process calc -PassThru).Id
Start-Process calc -PassThru | Get-UIAWindow

There should be placed a note: whereas the asterisk ‘*’ is a replacement for anything including an empty place, the question mark ‘?’ requires that something non-empty was under the sign. I.e., the following replacements are right:
Calc*
*Calculator*
C*l*l*
However, Cal?culator is not right. Ca?culator or Cal?ulator both are right.

Daily automation: eliminating the fragility of tests


Why do test fail? It’s the question every automated tester investigated into. Today we’ll discuss unexpected changes in the software under tests. This topic is hardly relevant to developers or even to testers in projects where no inexplicable changes allowed (have you seen any?).

When a new build is baked, and all nobody-knows-what changes are baked in, your tests might not find something vital and omit a great amount of the product functionality or stop completely.

One of solutions we accepted is to enable the search using wildcards. To our surprise, the wildcard search within a not great number of controls works with the same speed as the exact search. Wildcard search within a great number of controls is slower than exact search, possibly the latter uses caching or some hidden indices that we can’t use. There is a place for further investigation. In case of the fruitless search, both searches use almost the same time, the time the timeout told.

Task: use UIAutomation to get a control by a fragment of its name.

Requirements: compare times that needed for exact-match search and wildcard search.

Solution: below is the script that runs a process, gets its window, gets a control by exact match or using wildcard and reports the time used (part of the samples package) :

Set-StrictMode -Version Latest;
ipmo [path]\UIAutomation.dll;
[UIAutomation.Preferences]::OnSuccessDelay = 0;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching (one of few controls of the MenuItem type):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name File}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching (one of few controls of the MenuItem type):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name Fil*}).TotalSeconds
Stop-Process -Name mmc;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching (one of a heap of controls of the Edit):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name BranchCache}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching (one of a heap of controls of the Edit):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name Branch*}).TotalSeconds
Stop-Process -Name mmc;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching (one of few controls of the MenuItem type):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name File}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching (one of few controls of the MenuItem type):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name *il?}).TotalSeconds
Stop-Process -Name mmc;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching (one of a heap of controls of the Edit):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name Workstation}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching (one of a heap of controls of the Edit):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name ?orksta*i*n*}).TotalSeconds
Stop-Process -Name mmc;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching (one of few controls of the MenuItem type - it takes the full time timeout is set to):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name nothing}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching (one of few controls of the MenuItem type - it takes the full time timeout is set to):";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAMenuItem -Name not?ing}).TotalSeconds
Stop-Process -Name mmc;

# regular search
[UIAutomation.Preferences]::DisableExactSearch = $false
Write-host "exact matching:";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name nothing}).TotalSeconds
Stop-Process -Name mmc;

# search with wildcards
[UIAutomation.Preferences]::DisableExactSearch = $true
Write-host "wildcard matching:";
(Measure-Command {Start-Process services.msc -PassThru | Get-UIAWindow | Get-UIAEdit -Name ?othing}).TotalSeconds
Stop-Process -Name mmc;

As can be seen, the search among controls that are not huge in number is so fast as exact matching. However, when it works among a significant number of control of the type chosen, search is slowing down. Why? Possibly, there caching is needed. No answer at the moment. the third part of the test script clearly shows that in an unhappy case both types of search fail at the same speed, after timeout expires.

Thus, for small applications or separate parts of a big one, there is the reason to use wildcard wherever it’s possible. Just to eliminate surprises your developers might easily involve you in.

%d bloggers like this: