Monthly Archives: August, 2012

0.8.0 Preview 5 is available to download


UIAutomation 0.8.0 Preview 5 has been uploaded to

http://uiautomation.codeplex.com/releases/view/93646 .

 

Even though Preview 4 was to some extent working, we fixed several bugs that we allowed to pass to Preview 4.

  • if no control was found, the Win32 API (FindWindowEx) returned the full collection of controls with a handle – fixed
  • in a rare case when UIAutomation couldn’t find a window, the search was additionally performed by calling FindWindow. This ability has been broken in 0.8.0P4 – fixed
  • output of some cmdlets (used rarely) was broken (i.e., a hashtable was outputted as a collection of items) – fixed
  • the logical bug (that was explained in the previous post) with the turbo timeout auto-switching only on a ‘no window’ error was fixed by adding the -IsCritical parameter to Get-UIA[ControlType] and Wait- cmdlets

Now, UIAutomation cmdlets with new output pass tests for UIAutomation 0.8.0P3 with the old, restricted output.

Advertisements

Daily automation: eliminating the fragility of tests. Part 3


Many of us faced the situation when a window is gone and a testing tool hits the air for long time. The testing tool tries to get a control, a next control, and so on. Every attempt takes time.

UIAutomation had the remedy for the problem for relatively long time. This feature has never been widely known, so that it’s time to tell you what it is and how it works.

Task: minimize the time that UIAutomation spends on controls of a window that failed to appear.

Requirements: demonstrate how to use the technique with default settings.

Solution: you don’t need to do anything to benefit from this technique in case if a window of your interest is gone:

ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow;
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout
Get-UIAWindow -Name calc1 # non-existing window
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout
Get-UIAWindow -Name calc* # this call get a window
[UIAutomation.Preferences]::Timeout
[UIAutomation.Preferences]::AfterFailTurboTimeout

After we ran this sample, we have the following output:
5000 – the default timeout
2000 – the AfterFailTurboTimeout
2000 – the timeout equals to AfterFailTurboTimeout. This is because calc1 has not been caught and the default timeout was considered as too long to wait for nothing
2000 – the AfterFailTurboTimeout
5000 – the default timeout was restored after the test have got a window
2000 – the AfterFailTurboTimeout
Note: this funcitonality has been broken in 0.8.0 Preview 4 (more accurately, 0.8.0P4 was released despite the fact that several unit tests have not been fixed. Just because this functionality is not widely known and never used explicitly).

Internally, UIAutomation checks that [UIAutomation.CurrentData]::CurrentWindow is $null and [UIAutomation.CurrentData]::LastResult also is $null.

Well, what about controls? Imagine the case you have a window and a child window or a control failed to appear.

This problem is not so obvious as the ‘no such window’ problem. ‘One control is gone’ is a typical case during the development cycle. Nonetheless, some controls are vital for the whole test. For this, a new parameter is here: -IsCritical. Let’s see how it works:

ipmo [path]\UIAutomation.dll
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 10
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 10 -IsCritical
[UIAutomation.Preferences]::Timeout
[UIAutomation.CurrentData]::LastResult
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1
[UIAutomation.Preferences]::Timeout
Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name 1 -IsCritical
[UIAutomation.Preferences]::Timeout

Below is the output:
5000 – the default timeout
5000 – the default timeout as the -IsCritical parameter has not been used
2000 – the turbo timeout (thanks to the -IsCritical parameter)
5000 – the default timeout (after the first control that has been caught successfully)
5000 – IsCritical does nothing if there wasn’t a problem

The general rule is to use the -IsCritical parameter for child windows, default buttons and controls like combo boxes and radio buttons if their state blocks you from going forward.

0.8.0 Preview 4 is available to download


UIAutomation 0.8.0 Preview 4 has been uploaded to http://uiautomation.codeplex.com/releases/view/93118 and now you can try new output.

Except a couple of issues in the functionality that is not widely used, all our unit tests are green. Nonetheless, we’d recommend you to play with this release with some care: there might be combinations of code where new cmdlets may behave slightly differently.

For example,

Get-UIAWindow ... | Get-UIAButton ... | %{ $_ | Read-UIAControlName ... }

with this release can be written as

Get-UIAWindow ... | Get-UIAButton ... | Read-UIAControlName ...

 

0.8.0 Preview 4: no more restrictions on pipeline


In response to a request on the project forum, UIAutomation now supports outputting all the items that engine returns. How did it work earlier? Get-UIAWindow returned only one window, so did Get-UIA[ControlType] cmdlets.

How the most of traditional cmdlets work? Get-Process returns all the processes that match or simply all processes in the system. Get-ChildItem returns all the file system items in the current folder or wherever the -Path parameter said, if no filter supplied. Even Selenium cmdlets return all web elements that match a condition.

Thus, to meet standard practices, UIAutomation now returns all what match. Get-UIAWindow returns all ‘Calculator’ windows if the command a user issued was

Get-UIAWindow -Name calc*

Note that the [UIAutomation.CurrentData]::CurrentWindow variable (i.e. the current window is the last window object in the pipeline).
Get-UIAButton returns all buttons of the window if the least restrictive wildcard is given:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name *


Pattern cmdlets (Invoke-, Get- and Set-) and Read- cmdlets accept such output. Even Get-UIA[ControlType] -Win32 parameter returns not the only object as earlier but all that match. Note that -Win32 does not understand types of controls and returns all whose names match the condition:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name * -Win32 | Read-UIAControlName


The sample outputs names of all controls with handles.

Finally, if some of you readers wanted to click all the buttons at a time (I wanted), here is the script:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -Name * | ?{ ($_ | Read-UIAControlName) -ne 'Close' } | Invoke-UIAButtonClick

Web automation: a small data-driven test with Selenium


Just to observe, how Selenium cmdlets can be working ‘in parallel’:

ipmo [path]\SePSX.dll

$searchData = @("cheese", "juice", "wine");

[SePSX.Preferences]::OnSuccessDelay=0
[SePSX.Preferences]::OnErrorDelay=0
[SePSX.Preferences]::OnSleepDelay=10
[SePSX.Preferences]::Highlight=$false
[SePSX.Preferences]::HighlightParent=$false

[int]$counter = 0;
$drivers = Start-SeChrome -Count 3;
$drivers | Enter-SeURL "http://google.com" | %{ $null = $_ | Get-SeWebElement -Name q | Set-SeWebElementKeys $searchData[$counter] | Submit-SeWebElement; $counter++; }

$drivers | Read-SeWEbDriverUrl;

Test Case Management: generating test results on the fly


The truth is that almost nobody writes test cases these days by hands. Okay, for big and serious, and requiring mind efforts things like scenarios, testers write. PowerShell frameworks are all about automation and demand as little manual work as possible. Like other GUI and Web testing tools, PowerShell frameworks generate test results by watching the code execution. Let’s go to the samples:

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

What did we expect from this code? We set the EveryCmdletAsTestResult setting on, started an instance of Firefox, navigated to Google and submitted a query. As we saw no errors, we want to get our passed test results. How to do that?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is as follows:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q
Status : PASSED

Name : Set-SeWebElementKeys Cheese
Status : PASSED

Name : Submit-SeWebElement;
Status : PASSED

Name :
Status : NOT TESTED
Every cmdlet reported itself as Passed, and five results display this. The sixth result is a pre-generated test result that will be used in the immediately next cmdlet call.

This is too ideal, now we change our code to obtain real-life results. We are seeking for one of Google 2.0 controls, namely ‘q2’ (Marissa Mayer’s gone, hasn’t she?).

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q2 | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

The code failed (controls have not been renamed yet), where are our results?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is below:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name :
Status : NOT TESTED

The third and the fourth results have different exceptions in their Descriptions, though it’s a place where module’s code review needed…

Now, let’s see how time was consumed:

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status,TimeSpent

As can be seen, there is a default 500 milliseconds’ delay after starting the browser, 2 seconds were spent on navigation to the google page, and 5 seconds (the full time of [SePSX.Preferences]::Timeout) were spent on attempts to get the control of our interest.

In practice, testers are often interested in Failed results, much more often that in the list of Passed. 🙂 How to obtain such a list? First of all, we need to import the TMX module. Selenium and UIAutomation modules use TMX indirectly, as a library, but the advanced functionality is available as cmdlets:

ipmo [path]\TMX.dll;
Search-TMXTestResult -FilterFailed | FL Name,LineNumber,Code,Details

If the time of every test result is stored, what benefits do we have? Consider using the following query:

Search-TMXTestResult -OrderByTimeSpent -Descending -FilterFailed | FL Name,TimeSpent,Code

This lists Failed test results and time consumed in the descending mode (for what kind of user the contemporary versions of MS Excel have been written? Maybe, I’m wrong here? Okay, I’ll rewrite). This command lists Failed test results from bigger time spent to smaller.

Web automation: SePSX 0.4.0


 

To the version 0.4.0, though it’s still alpha, the following features, best known as belonged to the UIAutomation framework, have been added:

  • On error delay
  • On success delay
  • Automated test result generation

On error and on success scriptblocks (actions), and logging are not completely done yet.

Among other features that have been implemented the last week, the following list:

  • Smart wait for an element ([SePSX.Preferences]::Timeout as you have already understood, by the analogy with UIAutomation)
  • The square over the element a cmdlet returned has been tied to the right position

What we do these days


Our readers may have imagined that the SoftwareTestingUsingPowerShell.com team is on vacation or got stuck in Selenium PowerShell eXtensions. The second thought would be partially right. Need to say that SePSX is not only our project at all and not only our concern now.

The goal of our current work is broader than just work for a couple of frameworks. What makes the UIAutomation cmdlets different from the majority of cmdlets? Here is the list:

  • post-cmdlet scriptblocks (actions) – on success
  • post-cmdlet scriptblocks (actions) – on error
  • automated writing of the test result, passed or failed
  • user-driven choice of what is the result
  • automated logging, including a fragment of code
  • painting a square around the outputted control, web element, etc
  • painting a square around its parent
  • automated management of the post-cmdlet delay, on success or on error

For this purpose, we are writing a library that provides a class derived from PSCmdlet. By deriving from this class, test cmdlets get all the functionality listed above.

When we finish, everybody will be able to create a PowerShell framework for a particular need with all testing capabilities we have in UIAutomation and SePSX modules.

PowerShell and Selenium go ahead of paid competitors


An interesting query can be seen at the indeed.com portal:

The demand for test automation engineers with QTP experience grows, whereas primordial WinRunner goes towards zero in popularity. TestComplete and Ranorex have never been more than outsiders, but there is also a surprise about TestManager.

I think that all non-enterprise test tools are simply less known to serious enterprise customers. The second cause may be here that an average HR manager does know TFS but not MTM.
Anyway, the strong marketing machine of Microsoft and the tight integration with Studio and TFS should help in MTM’s advancement.

Here, one hope takes place. As, first and foremost, a big ally of open source testing solutions, I’d like to predict that in several areas, namely Windows UI Automation, Web Automation, maybe Metro UI automation and, definitely, automation of labs of any kind, there will be a breakthrough of open source testing instruments.

Let’s see what we have now. Regarding Windows desktop automation, there are a lot of frameworks with the ability to test Win32, Windows Forms, WPF, Silverlight applications. The only thing to perform a step to the mass-market is the need in a covering application that would embrace frameworks to an out-of-the-box solution. I mean a typical ‘download, set up and run’ and ‘click’n’play’ solution.

There are problems with third-party controls though. Nonetheless, we can now create an end-to-end testing solution on bare .NET or PowerShell (even me, just a tester, created one of such kind).

In the world of web automation, there is the unbeatable leader now. Supported by almost all browser vendors, Selenium is getting a standard tool for web-testing (and even going to become an official standard).

Well, the state of affairs seems to be very good, even though paid competitors struggle with FOSS. I’d like to think that one or more vendors of paid testing software will drop a free solution to the public within next two years. I don’t know, of course, whether it will be MTM Express, TestIncomplete or Ranorex UltraLite, but I believe that outsiders will act in this way too.

Happy testing with free solutions!

Web automation: SePSX 0.3.0


Today, the version 0.3.0 has been uploaded.

%d bloggers like this: