Tag Archives: 0.8.0 Preview 1

0.8.0 Preview 1: UIARunner, UIAutomationSpy, and modules are published


Even though with some bugs and ugly-looking (it never was our intent to write GUI tools besides modules:)), UIARunner is published. As far as I know, this is the first GUI wrapper for PowerShell that looks like a software testing tool. ūüôā

This preliminary version supports:

  • running one script at a time
  • using the UIARunner.ps1 file (in the application’s folder) as a configuration file
  • generating test report on the fly (in the GUI)
  • generating¬†test report on the fly as a CSV file (both GUI and command-line versions produce such output)
  • unattended runs
  • the GUI version also shows how many test results are passed, failed and the velocity in newly introduced measures called ‘test results per second’ (trps)
  • of course, it supports basic sorting in the grid.

Known issues are numerous (again, this project lacks of professional GUI programmers :)):

  • bad multithreading
  • poor informing about what happened at the PowerShell level
  • and many, many lesser bugs.

Anyway, this version is worth experimenting with. Each package includes several simple scripts to try the runner. Scripts work similarly to other PowerShell hosts, note, however, that you need to use try/catch blocks to avoid terminating errors.

Advertisements

0.8.0 Preview 1: UIARunner for unattended tests


Today’s Preview 1 brings to us not only a new graphical test runner, it includes also command-line test runner: UIARunner for unattended runs.

Although this is a very preliminary version, it can be used to start clicking on your test app. How is this intended to work?

You simply run UIARunner.exe with one parameter, the path to a script with test code. Before¬†a blind run, you can run it in the graphical version of UIARunner: UIARunner.exe. Don’t be afraid of incompatibility! Both utilities are proven on it. They are compatible to the greatest extent, they even is one utility as well.

Being run without the parameter, it displays the form:

 

The form provides us with date/time, status, name and/or source code of test results, and paths to screenshots if there were generated ones.

The command-line version creates a csv file in the folder of the program.

As examples for GUI-based and unattended runs, you may use scripts from folder TMX¬†and UIARunner in the ‘samples’ package.

 

0.8.0 Preview 1: UIARunner as a graphical test tool


Today’s Preview 1 brings to us a new test runner: UIARunner for testing Metro UI and Windows UI.

This version, even though not a release, already can run a script and display test results. The current version supports only test results, i.e., events generated by the Close-TMXTestResult cmdlet or closed automatically through the parameter [UIAutomation.Preferences]::EveryCmdletAsTestResult set to $true.

The figure below shows how it works:

 

The picture was taken¬†when UIARunner¬†was working on a performance test: it should calculate the results of expressions 1 + 1 and 2 + 2 a thousand times, and thousand times try to find the 10 button (it’s a flaw that Microsoft didn’t manage to find more space to put this useful button on the form).

To obtain such a visual report, you need no more than start the application with default settings, open a script through the File -> Open menu item (or by pressing Ctrl+O) and run the script by clicking on the menu item Script -> Run or by pressing F5.

The application is not smooth and cool for its first version. Nonetheless, it already can be used for gathering script results.

Speaking about future improvements, the features that are planned to implement are:

  • ¬†supporting hierarchical reports based on test suites, test scenarios and test results and shown in a grouped list view
  • probably, pausing the script
  • maybe, some events or alerts on conditions like ‘too many failures’ or ‘global timeout interrupted the script’.

 

Test Case Management: filtering and sorting the results


Let’s start from yesterday’s script. I added the code that never would work: getting a non-existing window and finding the “10” button on Calculator. Here is the script:

ipmo [path]\TMX.dll
ipmo [path]\UIAutomation.dll
Start-Process calc
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n add | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton | Set-UIAFocus | Set-UIAControlKeys -Text "1{+}1{=}"

Get-UIAWindow -n "non-existing window" -Seconds 2
Get-UIAWindow -n calc* | Get-UIAButton -n 10 | Invoke-UIAButtonClick;

Stop-Process -Name calc

Search-TMXTestResult
Export-TMXTestResults -As HTML -Path c:\1\calc_test_results_generated.htm
c:\1\calc_test_results_generated.htm

The three lines copy-pasted from yesterday’s post do exactly what they did yesterday: generate thirteen test results. What do the lines I added? The first line tries to get the window that’s rarely seen in the real life (you may compile one, though). The next line searches for the 10 button (that Microsoft declined to add to the tool, with very oblique explanation).

Okay, what should we expect? The instruction ‘Get-UIAWindow -n “non-existing window” -Seconds 2′ fails after two seconds’ time, so does ‘Get-UIAWindow -n calc* | Get-UIAButton -n 10’.

What does it mean for the whole test? These test cases (i.e., commands) are considered as most time-consuming (several queries to the Automaiton tree until the timeout ends this) and, no doubts, failed.

All the test results that failed can be pulled out with the following code:

Search-TMXTestResult -FilterFailed

The worst test result, considering as the worst a test result with the greatest time spent, can be scooped up with the following piece of code;

Search-TMXTestResult -FilterFailed -OrderByTimeSpent -Descending

Test results that are generated automatically have names based on the code. This leads us to the opportunity to filter test results by a code snippet or a keyword:

Search-TMXTestResult -FilterNameContains Button

or

Search-TMXTestResult -FilterNameContains 10

All that described are going to be available today as part of 0.8.0. Preview 1.

Test Case Management: how to generate test cases from the code


Software engineers use test cases for serious testing. Test cases are written rules, written in words or in code, and intended to become test results. Test results reflect which test cases have passed and which have failed.

This means that test cases are theoretical, static part of testing, whereas test results are dynamic, practical part. Life cycle of a test case longs from several builds to several versions of the product the test case belongs to. Life cycle of test results is from run to run. Usually, a run is performed once per build. Sometimes, several runs (i.e., test results) are conducted per build.

Now, let’s take a look at how contemporary test tools publish results. Click’n’play tools generate results for every click. Is this valuable? I think so, because a test engineer often needs to investigate into ‘not clicked’ issue. To sum up, this is an industry-proven practice to write up test results automatically. In fact, nobody would write test cases like ‘click the Next button’ by hands.

Following the industry practices, we also added test results generation from the code. How it works? If the paramater [UIAutomation.Preferences]::EveryCmdletAsTestResult is set to $true, all the results brought to the pipeline will be also stored as test results.

Let’s take the following piece of code:

ipmo [path]\TMX.dll
ipmo [path]\UIAutomation.dll
[UIAutomation.Preferences]::EveryCmdletAsTestResult=$true
Start-Process calc
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n add | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton -n 1 | Invoke-UIAButtonClick;
Get-UIAWindow -n calc* | Get-UIAButton | Set-UIAFocus | Set-UIAControlKeys -Text "1{+}1{=}"

Search-TMXTestResult
Export-TMXTestResults -As HTML -Path c:\1\calc_test_results_generated.htm
c:\1\calc_test_results_generated.htm

In the code above, we can see thirteen calls of UIA cmdlets. These thirteen calls generate thirteen test results. That’s all, no more manual work.

The Search-TMXTestResult cmdlets outputs all the test results to the pipeline, allowing you to process them in your own way. The Export-TMXTestResults cmdlet creates a simple HTML report (the projects seriously lacks of a HTML/CSS programmer).

When this way to obtain test result would be right:

  • you are playing with a small experimental script and you simply don’t want to spend time for creating test cases to a probe script
  • your test suite is huge, so huge that writing test cases manually makes no sense
  • you are not a professional tester, but test results are valuable for you

0.8.0 Preview 1: ‘UIATools’ to split into two apps: UIAutomationSpy and UIARunner


Tomorrow’s 0.8.0 Preview 1 will introduce a new tool: UIARunner.exe. What is it and why we need more tools?

By now, UIAutomationSpy is overloaded¬†with tabs and other controls. Intended initially to be a ‘UISpy for PowerShell’ tool, it has been covered with a number of controls. The reason was sad and simple: there were found no open-source PowerShell host that would be easy to make Metro-enables (i.e. uiAccess’ible).

Okay, UIAutomationSpy has and will have the following features:

  • code for a control we are hovering over (i.e. Get-UIAButton -Name ‘buttonname’) in the topmost rectangle on the Code/Control tab
  • patterns list the control supports – the second rectangle
  • code sequence through the Automation tree in the lowest rectangle. The code will be shortened as it’s done in the Start-UIARecorder cmdlet
  • the family tree of the control on the tab Code/Hierarchy
  • properties of the control on the Code/Properties tab
  • Start and Stop buttons and a check box to put the code from the topmost rectangle to the clipboard
  • the Script tab for generated code and to be able to run a piece of code to learn how it works. The tool is not intended to rn test suites!
  • ‘Run all’, ‘(Run) Selection’ and ‘Break’ buttons for running the whole script or its part
  • the Results tab is for the results the script brought
  • the Tree tab is now unneccessary as the tree is shown on the Code/Hierarchy tab

Not all tabs are completed.

UIARunner should do the following:

  • to load the initialization script UIARunner.ps1 from the directory of the tool
  • to tun a test script without GUI: UIARunner.exe [path]\script.ps1
  • to run a test script in the GUI mode. The user can open a script through the open File dialog, run and break the execution from the main menu.
  • later, there will be also a progress screen with script results or errors

Here is also not all the bullets completed.

That’s the nearest roadmap.

 

 

 

%d bloggers like this: