Category Archives: Selenium module

Turned a year!


Just remembered that the PowerShell testing suite turned a year. As it believed, approx. on November 28th, 2011, the result of a programming mistake, of course (a piece of code didn’t work in PowerShell, just a usage mistake), the decision to write up C#-cmdlets has been made. 🙂

Since then, our frameworks have changed significantly. It was only UIAutomation which was the root of the suite. Now, there are several frameworks published and several versions upcoming.

Shortly, I’d drop a word about the Software Testing Using PowerShell team plans. From the features that are almost ready to those that are only in mind. What will the upcoming winter bring to us?

UIAutomation:

  • on fail, the Invoke-UIA[ControlType]Click cmdlets will try Win32 click by default. This behavior (the user wants it click and it clicks) is most awaiting (thanks to JohnQuest for the idea).
  • the Set-UIA[ControlType]Text cmdlets will check the result and try to use SendKeys on a failure
  • the Set-UIAControlKeys cmdlet will try several times to put text into the input field, checking the result
  • UIAutomationSpy are going to produce more useful code. Moreover, the code produced is going to be faster. How? When possible, UIAutomationSpy will set the -Win32 parameter, what, in certain situations, accelerates tests significantly (working with controls near grids and listviews is slow; this can be healed by using Win32 handles).
  • UIAutomationSpy will finally check code that it generates (sometimes what can be got by hovering over is not the same that can be got by searching in the Automation tree).
  • at least ten pages of documentation as what is published is like a mad mix
  • setup for the module
  • Java Access Brigde (that has not been tried yet)
  • we will try to use UIA2 if it’s possible to combine two UI Automation technologies

SePSX:

  • support for ChromeOptions, InternetExplorerOptions and FirefoxProfile
  • fast-working ‘named’ cmdlets Get-Se[TagName]
  • cmdlets for working with tables
  • SeleniumSpy
  • in a perspective, support for other browsers

TMX:

  • TestLink add-in (missing functionality)
  • TFS add-in
  • working with test cases in SQLite, TestLink, various files

AWSPSX:

  • EC2 and S3 cmdlets to allow automated test lab deployment and execution

ESXiMgmt:

  • a small set of cmdlets allowing us to start/stop/suspend/snapshot guests

Internals:

  • these days we are working on harnessing a Dependency Injection framework. By now, SePSX is partially sitting on Autofac 2.6 and we are also experimenting with NInject 3 in other projects. TLAddin has benefited from using Moq. The reasons we are working on not a business-understood features are such:
  1. the number of tests is growing and the testing cycle should be running every several minutes. Hundreds of tests we already have are for daily framework verification (they take twenty minutes or more, what is critical for us). On the opposite, a hundred of unit tests we have already had take less than five seconds (and around seventeen seconds does Gallio take to get the consciousness at the start of testing)
  2. we need more through-out testing of our code
Advertisements

Projects’ roadmap’s gotten some dates


The recently published roadmap now contains schedule for upcoming functionality. As time continues, more features’ll get exact dates where only month is written now.

Web automation: a small data-driven test with Selenium


Just to observe, how Selenium cmdlets can be working ‘in parallel’:

ipmo [path]\SePSX.dll

$searchData = @("cheese", "juice", "wine");

[SePSX.Preferences]::OnSuccessDelay=0
[SePSX.Preferences]::OnErrorDelay=0
[SePSX.Preferences]::OnSleepDelay=10
[SePSX.Preferences]::Highlight=$false
[SePSX.Preferences]::HighlightParent=$false

[int]$counter = 0;
$drivers = Start-SeChrome -Count 3;
$drivers | Enter-SeURL "http://google.com" | %{ $null = $_ | Get-SeWebElement -Name q | Set-SeWebElementKeys $searchData[$counter] | Submit-SeWebElement; $counter++; }

$drivers | Read-SeWEbDriverUrl;

Test Case Management: generating test results on the fly


The truth is that almost nobody writes test cases these days by hands. Okay, for big and serious, and requiring mind efforts things like scenarios, testers write. PowerShell frameworks are all about automation and demand as little manual work as possible. Like other GUI and Web testing tools, PowerShell frameworks generate test results by watching the code execution. Let’s go to the samples:

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

What did we expect from this code? We set the EveryCmdletAsTestResult setting on, started an instance of Firefox, navigated to Google and submitted a query. As we saw no errors, we want to get our passed test results. How to do that?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is as follows:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q
Status : PASSED

Name : Set-SeWebElementKeys Cheese
Status : PASSED

Name : Submit-SeWebElement;
Status : PASSED

Name :
Status : NOT TESTED
Every cmdlet reported itself as Passed, and five results display this. The sixth result is a pre-generated test result that will be used in the immediately next cmdlet call.

This is too ideal, now we change our code to obtain real-life results. We are seeking for one of Google 2.0 controls, namely ‘q2’ (Marissa Mayer’s gone, hasn’t she?).

ipmo [path]\SePSX.dll
[SePSX.Preferences]::EveryCmdletAsTestResult = $true;
Start-SeFirefox | Enter-SeURL "http://google.com" | Get-SeWebElement -Name q2 | Set-SeWebElementKeys Cheese | Submit-SeWebElement;

The code failed (controls have not been renamed yet), where are our results?

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status

The output is below:
Name : Start-SeFirefox
Status : PASSED

Name : Enter-SeURL “http://google.com”
Status : PASSED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name : Get-SeWebElement -Name q2
Status : FAILED

Name :
Status : NOT TESTED

The third and the fourth results have different exceptions in their Descriptions, though it’s a place where module’s code review needed…

Now, let’s see how time was consumed:

[TMX.TestData]::CurrentTestScenario.TestResults | FL Name,Status,TimeSpent

As can be seen, there is a default 500 milliseconds’ delay after starting the browser, 2 seconds were spent on navigation to the google page, and 5 seconds (the full time of [SePSX.Preferences]::Timeout) were spent on attempts to get the control of our interest.

In practice, testers are often interested in Failed results, much more often that in the list of Passed. 🙂 How to obtain such a list? First of all, we need to import the TMX module. Selenium and UIAutomation modules use TMX indirectly, as a library, but the advanced functionality is available as cmdlets:

ipmo [path]\TMX.dll;
Search-TMXTestResult -FilterFailed | FL Name,LineNumber,Code,Details

If the time of every test result is stored, what benefits do we have? Consider using the following query:

Search-TMXTestResult -OrderByTimeSpent -Descending -FilterFailed | FL Name,TimeSpent,Code

This lists Failed test results and time consumed in the descending mode (for what kind of user the contemporary versions of MS Excel have been written? Maybe, I’m wrong here? Okay, I’ll rewrite). This command lists Failed test results from bigger time spent to smaller.

Web automation: SePSX 0.4.0


 

To the version 0.4.0, though it’s still alpha, the following features, best known as belonged to the UIAutomation framework, have been added:

  • On error delay
  • On success delay
  • Automated test result generation

On error and on success scriptblocks (actions), and logging are not completely done yet.

Among other features that have been implemented the last week, the following list:

  • Smart wait for an element ([SePSX.Preferences]::Timeout as you have already understood, by the analogy with UIAutomation)
  • The square over the element a cmdlet returned has been tied to the right position

PowerShell and Selenium go ahead of paid competitors


An interesting query can be seen at the indeed.com portal:

The demand for test automation engineers with QTP experience grows, whereas primordial WinRunner goes towards zero in popularity. TestComplete and Ranorex have never been more than outsiders, but there is also a surprise about TestManager.

I think that all non-enterprise test tools are simply less known to serious enterprise customers. The second cause may be here that an average HR manager does know TFS but not MTM.
Anyway, the strong marketing machine of Microsoft and the tight integration with Studio and TFS should help in MTM’s advancement.

Here, one hope takes place. As, first and foremost, a big ally of open source testing solutions, I’d like to predict that in several areas, namely Windows UI Automation, Web Automation, maybe Metro UI automation and, definitely, automation of labs of any kind, there will be a breakthrough of open source testing instruments.

Let’s see what we have now. Regarding Windows desktop automation, there are a lot of frameworks with the ability to test Win32, Windows Forms, WPF, Silverlight applications. The only thing to perform a step to the mass-market is the need in a covering application that would embrace frameworks to an out-of-the-box solution. I mean a typical ‘download, set up and run’ and ‘click’n’play’ solution.

There are problems with third-party controls though. Nonetheless, we can now create an end-to-end testing solution on bare .NET or PowerShell (even me, just a tester, created one of such kind).

In the world of web automation, there is the unbeatable leader now. Supported by almost all browser vendors, Selenium is getting a standard tool for web-testing (and even going to become an official standard).

Well, the state of affairs seems to be very good, even though paid competitors struggle with FOSS. I’d like to think that one or more vendors of paid testing software will drop a free solution to the public within next two years. I don’t know, of course, whether it will be MTM Express, TestIncomplete or Ranorex UltraLite, but I believe that outsiders will act in this way too.

Happy testing with free solutions!

Web automation: SePSX 0.3.0


Today, the version 0.3.0 has been uploaded.

%d bloggers like this: