Monthly Archives: July, 2012

Web automation: even more PowerShell for getting a WebElement


Yesterday, we worked on a translation of the sample from SeleniumHQ to PowerShell. One of the great PowerShell advantages is the ability to use .NET objects in code similarly to what C# programmers do. I’m speaking about the following piece of code:

$ff01 = Start-SeFirefox;
$searchBox = ($ff01 | Enter-SeURL -URL "http://www.google.com/" | Get-SeWebElement -Name "q");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ff01.Title;
$ff01 | Stop-SeFirefox;

We have been using two variables here, $ff01 for a browser instance (the driver) and $searchBox for the prominent Google search text box. Using methods .SendKeys(text), .Submit() and properties like .Title is what is considered by purists as the ‘CSharp style’. The purists (they are also known for the abbreviation MVP. I really don’t know how they managed to shorten the word ‘purist’ to ‘MVP’ :)) state that the only right way of using PowerShell is end-to-end pipelining. Okay, today’s our efforts are put in this direction:

$ff01 = Start-SeFirefox;
$searchBox = ($ff01 | `
 Enter-SeURL -URL "http://www.google.com/" | `
 Get-SeWebElement -Name "q" | `
 Set-SeWebElementKeys -Text "Cheese" | `
 Submit-SeWebElement);

Write-Host "Text:";
$searchBox | Read-SeWebElementText
Write-Host "Enabled:";
$searchBox | Read-SeWebElementEnabled
Write-Host "Displayed:";
$searchBox | Read-SeWebElementDisplayed
Write-Host "Selected:";
$searchBox | Read-SeWebElementSelected
Write-Host "TagName:";
$searchBox | Read-SeWebElementTagName
Write-Host "Size:";
$searchBox | Read-SeWebElementSize
Write-Host "Location:";
$searchBox | Read-SeWebElementLocation

sleep -Seconds 3; # to observe the result
$ff01.Title;
$ff01 | Stop-SeFirefox;

This time, all the code working with an WebElement is put through the pipeline.

Web automation: starting a browser and getting an element


Testing of web sites always required a lot of small tests. UI Automation is not good there due to the following flaws:

  • it’s slow. The more windows, tabs or elements are given, the slower UI Automation is
  • it can’t get a range of elements. The¬†UIA COM wrapper can more, but for now it is not good at patterns
  • it is not cross-browser. Whereas Internet Explorer and Firefox are seen as a set of UI Automation controls, WebKit browsers are often sets of tabs in a window.

These problems usually led testers to using such instruments as Selenium or watir.

Nonetheless, things are not so bad for PowerShell testers as it seems to! There is no strict need to write all the test code in CSharp-like style, on the contrary, continue using pipelines:

$ff01 = Start-SeFirefox;
$searchBox = ($ff01 | Enter-SeURL -URL "http://www.google.com/" | Get-SeWebElement -Name "q");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ff01.Title;
$ff01 | Stop-SeFirefox;

This is nothing else than the sample the Selenium project provides:

using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;

// Requires reference to WebDriver.Support.dll
using OpenQA.Selenium.Support.UI;

class GoogleSuggest
{
    static void Main(string[] args)
    {
        // Create a new instance of the Firefox driver.

        // Notice that the remainder of the code relies on the interface,
        // not the implementation.

        // Further note that other drivers (InternetExplorerDriver,
        // ChromeDriver, etc.) will require further configuration
        // before this example will work. See the wiki pages for the
        // individual drivers at http://code.google.com/p/selenium/wiki
        // for further information.
        IWebDriver driver = new FirefoxDriver();

        //Notice navigation is slightly different than the Java version
        //This is because 'get' is a keyword in C#
        driver.Navigate().GoToUrl("http://www.google.com/");

        // Find the text input element by its name
        IWebElement query = driver.FindElement(By.Name("q"));

        // Enter something to search for
        query.SendKeys("Cheese");

        // Now submit the form. WebDriver will find the form for us from the element
        query.Submit();

        // Google's search is rendered dynamically with JavaScript.
        // Wait for the page to load, timeout after 10 seconds
        WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
        wait.Until((d) => { return d.Title.ToLower().StartsWith("cheese"); });

        // Should see: "Cheese - Google Search"
        System.Console.WriteLine("Page title is: " + driver.Title);

        //Close the browser
        driver.Quit();
    }
}

Not surprisingly, the PowerShell code is shorter, prettier and looks comprehensible. Need to port to another browser? It’s easy (enough). The code below does the same in three browsers and, moreover, in two search engines:

$ff01 = Start-SeFirefox;
$searchBox = ($ff01 | Enter-SeURL -URL "http://www.google.com/" | Get-SeWebElement -Name "q");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ff01.Title;
$ff01 | Stop-SeFirefox;

$ch01 = Start-SeChrome;
$searchBox = ($ch01 | Enter-SeURL -URL "http://www.google.com/" | Get-SeWebElement -Name "q");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ch01.Title;
$ch01 | Stop-SeChrome;

$ie01 = Start-SeInternetExplorer;
$searchBox = ($ie01 | Enter-SeURL -URL "http://www.google.com/" | Get-SeWebElement -Name "q");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ie01.Title;
$ie01 | Stop-SeInternetExplorer;

$ff01 = Start-SeFirefox;
$searchBox = ($ff01 | Enter-SeURL -URL "http://www.yandex.ru/" | Get-SeWebElement -Id "text");
$searchBox.SendKeys("Cheese");
$searchBox.Submit();
sleep -Seconds 3; # to observe the result
$ff01.Title;
$ff01 | Stop-SeFirefox;

Test web sites with pleasure!

BGShell for Metro UI testing


Today I built an experimental version of BGShell. It allows you to run cmdlets hovering over the Metro UI. BGShel also support some intellisence and shows objects’ properties.

The package is here.
A fly in the oinment is that it’s built for .NET 3.5. Nonetheless, all what you need to do is to agree with your test Windows 8 when it offer to download .NET 2.0/3.5.

Metro automation: navigating to a certain page


A typical Metro UI application is a set of pages. One is main, and there are several pages where we can go from the main page.

A Metro UI app is unlikely to a traditional app, which often starts from some starting point. A Metro UI app starts for the first time with its main page, but further is may start from wherever page the user left the app.

In our sample, we’ll be using the *Wizard* cmdlets. What are they?

1. the New-UIAWizard cmdlet creates a wizard object, provides it with name and contains a code to start the AUT (application under test), for example. Some initial code.

2. the Add-UIAWizardStep cmdlet creates a step, provides the step with a name, takes a wizard object as an input.

First action we should do on a step is to examine the step. The -SearchCriteria parameter does exactly this task.

After that, we run code for this step. There are two actions we can use, for forward movement and for backward. In out Metro UI app we will be using only forward steps for simplicity sake.

3. the Invoke-UIAWizard cmdlets runs the wizard (in fact, the code we added to the New-UIAWizard cmdlet). It’s usually used for starting the AUT. For purposes of Metro UI testing, we will use this action to show the Start screen and to open the app window.

4. the Step-UIAWizard cmdlet checks the -SearchCriteria (i.e., checks that controls are here and we are on the right page) and runs the step code.


Set-StrictMode -Version Latest

[UIAutomation.Preferences]::Timeout = 10000;
[UIAutomation.Preferences]::EveryCmdletAsTestResult = $true;
[UIAutomation.Preferences]::OnSuccessDelay = 300;

# names of the pages and buttons in the app menu
[string]$constPageMain = 'Home';
[string]$constPageFeaturedDestinations = 'Featured Destinations';
[string]$constPageDestinations = 'Destinations';
[string]$constPageFlights = 'Flights';
[string]$constPageHotels = 'Hotels';
[string]$constPageBestOfWeb = 'Best Of Web';

# names of wizard steps
[string]$constWizardName = 'TravelWizard';
[string]$constWizardStepHome = 'Home';
[string]$constWizardStepDestinations = 'Destinations';
[string]$constWizardStepFlights = 'Flights';
[string]$constWizardStepHotels = 'Hotels';
[string]$constWizardStepBestOfWeb = 'BestOfWeb';

# we need to clear the collection of wizards or delete
# the wizard we are playing with.
# Otherwise, the next run will throw an exception
# 'Wizard already exists'
[UIAutomation.WizardCollection]::Wizards.Clear();

# Creating a new wizard
New-UIAWizard -Name $constWizardName `
 -StartAction {
 # setting the Metro Start screen
 Get-UIADesktop;
 Show-UIAMetroStartScreen;

# clicking on the tile of the AUT
 Get-UIAListItem -Name 'Travel' | `
 Invoke-UIAListItemClick;

# the loading screen (if happened)
 try {
 Get-UIAWindow -Name 'Travel' | `
 Get-UIAProgressBar;
 #sleep -Seconds 5; # the app is loading
 }
 catch {}

 # start our tests from the Main page
 Show-UIAMetroMenu;
 Get-UIAWindow -Name 'Travel' | `
 Get-UIAMenuBar -Name 'App Bar' | `
 Get-UIAHyperlink -Name $constPageMain | `
 Invoke-UIAControlClick;
 } | `
 Add-UIAWizardStep -Name $constWizardStepHome `
 -SearchCriteria @{ControlType="Text";Name='Bing Travel'} `
 -StepForwardAction {
 #[System.Windows.Forms.MessageBox]::Show("StepHome");
 "<<<<<<<<<< On the Home page >>>>>>>>>>";
 try {
 Get-UIAText -Name 'Bing Travel';
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Home page >>>>>>>>>>" -TestPassed;
 }
 catch {
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Home page >>>>>>>>>>";
 }
 } -PassThru | `
 Add-UIAWizardStep -Name $constWizardStepDestinations `
 -SearchCriteria @{ControlType="Text";Name='Destinations'},@{ControlType="Text";Name='Region'} `
 -StepForwardAction {
 #[System.Windows.Forms.MessageBox]::Show("StepDestinations");
 "<<<<<<<<<< On the Destinations page >>>>>>>>>>";
 try {
 Get-UIAText -Name 'Destinations';
 Get-UIAText -Name 'Region';
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Destinations page >>>>>>>>>>" -TestPassed;
 }
 catch {
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Destinations page >>>>>>>>>>";
 }
 } -PassThru | `
 Add-UIAWizardStep -Name $constWizardStepFlights `
 -SearchCriteria @{ControlType="Text";Name='Flights'},@{ControlType="Text";Name='Schedule'} `
 -StepForwardAction {
 #[System.Windows.Forms.MessageBox]::Show("StepFlights");
 "<<<<<<<<<< On the Flights page >>>>>>>>>>";
 try {
 Get-UIAText -Name 'Flights';
 Get-UIAText -Name 'Schedule';
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Flights page >>>>>>>>>>" -TestPassed;
 }
 catch {
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Flights page >>>>>>>>>>";
 }
 } -PassThru | `
 Add-UIAWizardStep -Name $constWizardStepHotels `
 -SearchCriteria @{ControlType="Text";Name='Hotels'},@{ControlType="Text";Name='Check-in'} `
 -StepForwardAction {
 #[System.Windows.Forms.MessageBox]::Show("StepHotels");
 "<<<<<<<<<< On the Hotels page >>>>>>>>>>";
 try {
 Get-UIAText -Name 'Hotels';
 Get-UIAText -Name 'Check-in';
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Hotels page >>>>>>>>>>" -TestPassed;
 }
 catch {
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the Hotels page >>>>>>>>>>";
 }
 } -PassThru | `
 Add-UIAWizardStep -Name $constWizardStepBestOfWeb `
 -SearchCriteria @{ControlType="Text";Name='Travel'},@{ControlType="Text";Name='Explore'},@{ControlType="Text";Name='Plan a Trip'} `
 -StepForwardAction {
 #[System.Windows.Forms.MessageBox]::Show("Best Of Web");
 "<<<<<<<<<< On the 'Best Of Web' page >>>>>>>>>>";
 try {
 Get-UIAText -Name 'Travel';
 Get-UIAText -Name 'Explore';
 Get-UIAText -Name 'Plan a Trip';
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the 'Best Of Web' page >>>>>>>>>>" -TestPassed;
 }
 catch {
 Close-TMXTestResult -Name "<<<<<<<<<< Navigating to the 'Best Of Web' page >>>>>>>>>>";
 }
 };

# Start the wizard
[UIAutomation.Wizard]$wizard = Invoke-UIAWizard -Name $constWizardName;

# Click on the Destinations button in the app menu
Show-UIAMetroMenu;
Get-UIAWindow -Name 'Travel' | `
 Get-UIAMenuBar -Name 'App Bar' | `
 Get-UIAHyperlink -Name $constPageDestinations | `
 Invoke-UIAControlClick;

# Check that this is the Destinations page and run code (in the future sample)
$wizard | Step-UIAWizard -Name $constWizardStepDestinations

# Click on the Flights button in the app menu
Show-UIAMetroMenu;
Get-UIAWindow -Name 'Travel' | `
 Get-UIAMenuBar -Name 'App Bar' | `
 Get-UIAHyperlink -Name $constPageFlights | `
 Invoke-UIAControlClick;

# Check that this is the Flights page and run code (in the future sample)
$wizard | Step-UIAWizard -Name $constWizardStepFlights;

# Click on the Hotels button in the app menu
Show-UIAMetroMenu;
Get-UIAWindow -Name 'Travel' | `
 Get-UIAMenuBar -Name 'App Bar' | `
 Get-UIAHyperlink -Name $constPageHotels | `
 Invoke-UIAControlClick;

# Check that this is the Hotels page and run code (in the future sample)
$wizard | Step-UIAWizard -Name $constWizardStepHotels;

# Click on the 'Best Of Web' button in the app menu
Show-UIAMetroMenu;
Get-UIAWindow -Name 'Travel' | `
 Get-UIAMenuBar -Name 'App Bar' | `
 Get-UIAHyperlink -Name $constPageBestOfWeb | `
 Invoke-UIAControlClick;

# Check that this is the 'Best Of Web' page and run code (in the future sample)
$wizard | Step-UIAWizard -Name $constWizardStepBestOfWeb;

The dubious piece of code is getting a progress bar on loading the application. Sometimes, it is shown. Sometimes, there is no progress bar.

0.8.0 Preview 3: bug fix and improvements to UIARunner


UIARunner from 0.8.0 Preview 2 has a bug that is easy to work around, but It is difficult to find out this workaround.

The bug in fact is a combination of two bugs:

  • UIARunner failed to load its autoexec file called UIARunner.ps1 if the application is in a folder with spaces in the path, i.e., Program Files. There weren’t added quotes.
  • error reporting in the grid couldn’t display the right error

Even though it’s avoidable, it’s better to publish 0.8.0 Preview 3. Now, it works as

  • The UIARunner.ps1 autoexec file can be loaded from any folder
  • Binaries (UIAutomation.dll and TMX.dll) are loaded before the autoexec script
  • Grid and the report file provide an error’s description, for example, execution policy is set to Restricted.

0.8.0 Preview 2: wanted richer output?


Again and again, it’s not going to end. People ask me ‘why Metro UI testing doesn’t work from PowerShell ISE?’

PowerShell ISE and powershell.exe are Microsoft’s tools, they have no relation to Metro UI testing. Technically, Microsoft guys can build ISE and powershell.exe to make them uiAccess’ible. However, don’t wait this: why should they build a host for my cmdlets? To help compete with their Coded UI? I sooner believe that they will write their own UI cmdlets in v3 or v4 (as they usually answer to PowerGUI features, for example), than they do something specific for 3rd party cmdlets.

The other side of coin is that I’m not going to write one more code editor with debugging capabilities. I’d like to, but this is a time-consuming task. Not to say that an editor of appropriate quality is not so easy to write up.

That’s a dilemma: we can run PowerShell over Metro UI, but we can’t work deeply with variables and output.

0.8.0 Preview 2 solves this partly. Now, output is partially supported. The list of what is supported:

  • string output (code: “string”)
  • string output for objects (code: (Get-UIAWindow 0n calc*).Current | Out-String)

Write- cmdlets are not supported in this version. Possibly, they will be supported later.

Here is a sample:

[UIAutomation.Preferences]::EveryCmdletAsTestResult=$true
[UIAutomation.Preferences]::OnSuccessDelay = 0;

Start-Process calc
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
(Get-UIAButton -n add | Invoke-UIAButtonClick).Current.AutomationId;
(Get-UIAButton -n 1 | Invoke-UIAButtonClick).Current | Out-String;
Get-UIAWindow -n calc* | Get-UIAButton | Set-UIAFocus | Set-UIAControlKeys -Text "1{+}1{=}"
'Get-UIAWindow -n calc* | Out-String; ->'
Get-UIAWindow -n calc* | Out-String;
'(Get-UIAWindow -n calc*).Current | Out-String;'
(Get-UIAWindow -n calc*).Current | Out-String;

"now test should fail"
Get-UIAWindow -n "non-existing window" -Seconds 2
Get-UIAWindow -n calc* | Get-UIAButton -n 10 | Invoke-UIAButtonClick;

Output is shown as light-blue lines. It returns:

  • the button 1 (Invoke-UIAButtonClick returns)
  • the automation id of the button Add (again, the same cmdlet returns)
  • AutomationElementInfo (i.e., .Current). This is the button 1 again.
  • further, after an empty string, there are string lines to help you compare what means to put an AutomationElement and an AutomationElementInfo to the Out-String cmdlet.

It’s not proper way to debug your scripts, though it’s the only way for now.

0.8.0 Preview 2: double squares help to visualize controls better


Controls belong to a big hierarchy. What may be the reason to highlight only two levels of controls if, in practice, there might be ten or twenty levels in the hierarchy?

The pictures show what the double-highlighting is. The inner square is over the control of interest, whereas the circumscribing one is over its parent.

However, things suddenly change when you put the cursor over WPF applications, its (WPF’s) derivatives and HTML:

This is a rather typical case: we are seeing a button or a link and, at the same time, the topmost control is a text.

The same story in Metro UI applications:

Here it is seen even better:

and here too:

The name of the button is not ‘Permissions’ as we could think, the name is ‘Back’.

At the end, there is a brilliant example:

There are TWO things on top of the button! This is the out-of-the-box Mail application.

0.8.0 Preview 1: UIARunner, UIAutomationSpy, and modules are published


Even though with some bugs and ugly-looking (it never was our intent to write GUI tools besides modules:)), UIARunner is published. As far as I know, this is the first GUI wrapper for PowerShell that looks like a software testing tool. ūüôā

This preliminary version supports:

  • running one script at a time
  • using the UIARunner.ps1 file (in the application’s folder) as a configuration file
  • generating test report on the fly (in the GUI)
  • generating¬†test report on the fly as a CSV file (both GUI and command-line versions produce such output)
  • unattended runs
  • the GUI version also shows how many test results are passed, failed and the velocity in newly introduced measures called ‘test results per second’ (trps)
  • of course, it supports basic sorting in the grid.

Known issues are numerous (again, this project lacks of professional GUI programmers :)):

  • bad multithreading
  • poor informing about what happened at the PowerShell level
  • and many, many lesser bugs.

Anyway, this version is worth experimenting with. Each package includes several simple scripts to try the runner. Scripts work similarly to other PowerShell hosts, note, however, that you need to use try/catch blocks to avoid terminating errors.

0.8.0 Preview 1: UIARunner for unattended tests


Today’s Preview 1 brings to us not only a new graphical test runner, it includes also command-line test runner: UIARunner for unattended runs.

Although this is a very preliminary version, it can be used to start clicking on your test app. How is this intended to work?

You simply run UIARunner.exe with one parameter, the path to a script with test code. Before¬†a blind run, you can run it in the graphical version of UIARunner: UIARunner.exe. Don’t be afraid of incompatibility! Both utilities are proven on it. They are compatible to the greatest extent, they even is one utility as well.

Being run without the parameter, it displays the form:

 

The form provides us with date/time, status, name and/or source code of test results, and paths to screenshots if there were generated ones.

The command-line version creates a csv file in the folder of the program.

As examples for GUI-based and unattended runs, you may use scripts from folder TMX¬†and UIARunner in the ‘samples’ package.

 

0.8.0 Preview 1: UIARunner as a graphical test tool


Today’s Preview 1 brings to us a new test runner: UIARunner for testing Metro UI and Windows UI.

This version, even though not a release, already can run a script and display test results. The current version supports only test results, i.e., events generated by the Close-TMXTestResult cmdlet or closed automatically through the parameter [UIAutomation.Preferences]::EveryCmdletAsTestResult set to $true.

The figure below shows how it works:

 

The picture was taken¬†when UIARunner¬†was working on a performance test: it should calculate the results of expressions 1 + 1 and 2 + 2 a thousand times, and thousand times try to find the 10 button (it’s a flaw that Microsoft didn’t manage to find more space to put this useful button on the form).

To obtain such a visual report, you need no more than start the application with default settings, open a script through the File -> Open menu item (or by pressing Ctrl+O) and run the script by clicking on the menu item Script -> Run or by pressing F5.

The application is not smooth and cool for its first version. Nonetheless, it already can be used for gathering script results.

Speaking about future improvements, the features that are planned to implement are:

  • ¬†supporting hierarchical reports based on test suites, test scenarios and test results and shown in a grouped list view
  • probably, pausing the script
  • maybe, some events or alerts on conditions like ‘too many failures’ or ‘global timeout interrupted the script’.

 

%d bloggers like this: