Tag Archives: Windows 8

Metro automation: the first step


Visit our Metro UI automation page.

Yesterday, a customer was disappointed with the difficulty of installing the module. Although the UIAutomation module wasn’t guilty, it would be better to figure out how to install the module on your Windows 8 or Windows Server 2012 host.

First of all, download the right package. Go to the project page and click the Downloads tab:

There, you need to download the package that is called as ‘UIAutomation X.Y.Z for Metro testing’:

Download the package with name like ‘UIAutomation.X.Y.Z.for.Metro.testing.zip’ and open it. Don’t put the content directly to the %ProgramFiles% folder! It won’t work. Security has been tightened even more (and it’s a good news for us), so that we need to do some preparations. Open the package where it was saved (supposedly, in your Downloads folder):

unpack the zip into your user’s folder, for example, Documents. I created the folder ‘Metro’ and paste the files to there. Open properties of dlls:

Now, you need to unblock the binaries one by one::

Eventually, you can copy all the files to a folder within the ‘Program Files’ hierarchy. If you had not unblocked the binaries, or tried to unblock in the ‘Program Files’ folder, your output in PowerShell console would look like that:

If you did all I wrote above, you output will be much cleaner:

The next step is to install the certificate as described here.

Advertisements

Release: version 0.7.12 that does work with Metro UI from UIAutomationSpy a bit more comfortable is out


Visit our Metro UI automation page.

Today, three hosts (Windows 8 RP x64, Windows 8 RP x86, Windows Server 2012 RC) agreed on that the version 0.7.12 is the version we can start with.
What we now can do for testing Metro UI applications?

  • we can start UIAutomationSpy manually and type or paste script code into it, or type a path to the script file. Automated run with preloaded code or script file is not done yet
  • we can show the Start screen (the Show-UIAMetroStartScreen cmdlet) from UIAutomationSpy
  • we can call charm (the Show-UIAMetroCharm cmdlet)
  • we also can call the app/ the Start screen menu (the Show-UIAMetroMenu cmdlet)
  • we can select an app at the Start screen (the Get-UIAListItem cmdlet issuing somewhat after the Get-UIADesktop cmdlet, as the Get-UIA[ControlType] cmdlets require input object)
  • we can start an app by sending an InvokePattern-click to it (the Invoke-UIAListItemClick cmdlet)

All that is mentioned above can be done with the version 0.7.12 and using existing blog posts.

Note: the UIASendKeys utility is deprecated. the Invoke-UIAHotkey is not recommended for use and will be rewritten soon.

Download 0.7.12 from here, the samples.

Metro automation: a nightmare of code generation


Visit our Metro UI automation page.

 

I’ve been struggling with code generation in UIAutomationSpy for a couple of hours. What’s the problem?

Code that UIAutomationSpy generates for Windows applications is what you can run immediately after recording (okay, some polishing or additions may be needed). At least, from today, when I removed the first line of code (i.e., RootElement) and UIAutomaitonSpy is now so close to the Start-UIARecorder-generated code as it might be.

For example, this is code for Notepad (an emtpy document, I click File, the dialog opens and I close it):

Get-UIAWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

Get-UIAMenuBar -AutomationId 'MenuBar' -Name 'Application' | `

Get-UIAMenuItem -AutomationId 'Item 1' -Name 'File'
Get-UIAWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

Get-UIAMenuBar -AutomationId 'MenuBar' -Name 'Application' | `

Get-UIAMenuItem -AutomationId 'Item 1' -Name 'File' | `

Get-UIAMenu -Class '#32768' -Name 'File' | `

Get-UIAMenuItem -AutomationId 'Item 1' -Name 'New'
Get-UIAWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

Get-UIADocument -AutomationId '15' -Class 'Edit'
Get-UIAWindow -Class 'Notepad' -Name 'Untitled - Notepad' | `

Get-UIAWindow -Class '#32770' -Name 'Open' | `

Get-UIATitleBar -AutomationId 'TitleBar' -Name 'Open'

The code is readable (to some extent), though even such code should be prepared to run well.

In fact, the code is raw and the Start-UIARecorder cmdlet still works better:

Get-UIAWindow -n *notepad* | Get-UIAMenuItem -Name File | Invoke-UIAMenuItemExpand | Get-UIAMenuItem -Name Open* | Invoke-UIAMenuItemClick

Anyway, a Windows app is not a mythical beast. Metro UI is more complicated; below is a typical piece of generated code:

Get-UIAList -AutomationId 'GridRoot' -Class 'GridRoot' -Name 'Start Apps' | `
Get-UIAPane -AutomationId 'ScrollViewer' -Class 'TouchScrollViewer' | `
Get-UIAPane -AutomationId 'content' -Class 'GridContent' | `
Get-UIAGroup -AutomationId 'Group 2305843009213693952' -Class 'GridGroup' -Name 'Mail' | `
Get-UIAListItem -AutomationId 'microsoft.windowscommunicationsapps_8wekyb3d8bbwe!Microsoft.WindowsLive.Chat' -Class 'GridTileElement' -Name 'Messaging'

This code simpy finds the Messaging app. This is a dead code. To make the code alive, we need:

  1. add Get-UIADesktop before the first instruction
  2. add Show-UIAMetroStartScreen before the first instruction
  3. remove the line 2: Get-UIAPane -AutomationId ‘ScrollViewer’ -Class ‘TouchScrollViewer’ | `
  4. remove the line 3: Get-UIAPane -AutomationId ‘content’ -Class ‘GridContent’ | `
  5. remove the part of line 5: -AutomationId ‘microsoft.windowscommunicationsapps_8wekyb3d8bbwe!Microsoft.WindowsLive.Chat’ -Class ‘GridTileElement’

After editing, code looks like:

Get-UIADesktop | Get-UIAList -AutomationId 'GridRoot' -Class 'GridRoot' -Name 'Start Apps' | `
Get-UIAGroup -AutomationId 'Group 2305843009213693952' -Class 'GridGroup' -Name 'Mail' | `
Get-UIAListItem -Name 'Messaging'

Finally, the right code would be (for Windows 8 RP):

Get-UIADesktop;
Show-UIAMetroStartScreen;
Get-UIAListItem -Name *Internet*Explorer* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *people* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *messag* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *phot* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *calen* | Invoke-UIAListItemClick;

and for Windows Server 2012:

Get-UIADesktop;
Show-UIAMetroStartScreen;
Get-UIAListItem -Name *Internet*Explorer* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *control* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *computer* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *task*man* | Invoke-UIAListItemClick;

Show-UIAMetroStartScreen;
Get-UIAListItem -Name *admi*tool* | Invoke-UIAListItemClick;

As can be seen, the problem is that code UIAutomationSpy generates is far away of what is working…

The worst of all, the code for Windows applications can be useful from the top control to the current, whereas the code for Metro UI apps should be filtered, edited and some commands should be added to it.
This requires two branches of code generation, doesn’t it?

There is a video based on code samples (HD mode is recommended if you want to read the code).

function Start-MetroApp
{
param( [string]$appName )

Show-UIAMetroStartScreen;
Get-UIAListItem -Name $appName | Invoke-UIAListItemClick;
}
Get-UIADesktop;

Start-MetroApp Music;

Start-MetroApp Finan*;

Start-MetroApp '*internet*explorer*';

Start-MetroApp people;

Metro automation: UIAutomationSpy displays even more


Visit our Metro UI automation page. This post is deprecated.

 

The version 0.7.11 is out specially for Metro UI testers and developers. There are a couple of improvements to UIAutomationSpy:

  • now you can see the full path to a control in terms of the UIAutomation module.
  • you can see the same graphically. Okay, just as a tree.
  • you do not need UISendKeys anymore. It simply does not ship. Use the Invoke-UIAHotKey cmdlet as shown on the picture
  • access the charm? Not a problem! 
  • finally, how to pull the application menu? Send Win+Z to the Start screen  or to an application:

Happy testing!

P.S. All the faces on tiles belong to their respective owners.

Metro automation: now, UIAutomationSpy displays more


Visit our Metro UI automation page.

 

I’ve been working on a material for upcoming posts about Metro UI automation and found that it’s often inevident from where a control grows. I mean that any information about the immediate parent and the ancestry at all is a necessity for writing scripts.

With that said, a new version of UIAutomationSpy (sooner a half-version as it’s not the time to publish such small changes as a version) would be accepted by the public well.

Right now there is a version that displays the immediate parent’s code along with the code of a control you are observing.

Here we are hovering over the Start screen. The full path to a tile is shown in the lower rectangle:

The picture on the lower-left tile is a property of http://echo.msk.ru/. It shows that the president, the prime minister and the main speaker of the upper part of Duma are from the city I live in.  Above we’ve hovering over People, now we are hovering over Travel:

The upper-left tile informs us that I have a report on Family Safety. This is a report for an account I’m not related to in any respect. How to spy on somebody you even don’t know with the help from Microsoft, read here.

What is the main part of the Travel application made of?

After we opened the charm (Win+C):

Settings:

Settings -> Feedback:

Dissecting a combo box:

Settings -> Share:

Settings -> Share -> Login to Evernote:

Finally, Settings -> Share -> Mail:

Don’t procrastinate the chance to test your Metro UI application, download it now.

Metro automation: getting the Start screen


Visit our Metro UI automation page. This post is deprecated

 

It might sound oddly, but the fact is that I’ve spent several hours in attempts to raise the Start screen. I tried moving the cursor over the lower left corner to get the ListItem, a small image of the Start screen. I tried to move cursor over the lower right corner and along the right side to get the desktop charm. I tries to use SendKeys to send Win (^{ESC}) or Win+C key combinations to the desktop. After all, I simply created an app that is running as another, non-privileged process and is able to send Ctrl+Esc. The solution looks overly complicated, nevertheless it works pretty well.

This was, as an example, one of my wrong solutions:


Get-UIADesktop | Get-UIAPane -Class 'Shell_TrayWnd' | Invoke-UIAControlClick -X 10 -Y 10 | Move-UIACursor -X -3 -Y 100;

Get-UIAListItem -Name 'Start screen' | Invoke-UIAListItemClick;

Okay, what is recommended:

1.) download the UIASendKeys.exe, placed separately for this time

2.) unpack it to a folder of your choice and run the application

It is to be run the whole test session.

3.) run UIAutomationSpy

4.) When you need to raise the Start screen (this can be needed frequently as it’s frightening of any movement over the desktop or flickering of applications that got some changes),  go to your working place, the Script tab, and type the following code:


Get-UIAWindow -n UIASendKeys | Get-UIAButton -Name Start* | Invoke-UIAButtonClick;

This raises the Start screen whenever you need it:

The idea behind this solution is the same that I used for testing of several products: run an application under test in another process and connect to it using inter-process communication (UIAutomation, in other words). This method is used in many test systems, NUnit is the first example that has been recalled.

A word of appreciation: Windows 8 accessibility performance improvements


Compared to runs on a Windows 7 x64 box as well as on a Windows 2008 SP2 x64 box, the same script from the post gave the great difference in the work with the FindAll call:

to get the AutomationElement corresponding to the Workstation service (at the bottom of the grid) through the FindFirst call takes the same time as to get this element by using FindAll with iteration through the collection.

If you ran the script on a Window 7 or Windows 2008 box, you probably noticed that the latter way is slower than the former as much  as several times.

On the opposite, on a Windows 8 CP x64 box and on a Windwos Server 8 Beta box, both times are almost equal.

Viva, Microsoft UI Automation team! You embedded something (I think it’s caching always ON) that greatly improved the performance of control collections.

%d bloggers like this: