Metro automation: the demo is published
The demo is published within the files of version 0.7.4 (though it’s almost 0.7.5).
The picture taken from a Windows Server 8 Beta host through the RDP connection:
Metro automation: the first script recorded!
With a bit of free time, I built a PowerShell host that works over Start Screen and recorded my first Metro script. As a host I used PoshConsole (a flamboyant shell, and I couldn’t use autocomplete for the UIAutomation cmdlets) with the manifest. At first, I tried to compile BGShell, but it falls each run, perhaps as a matter of using Windows Forms Aero for .NET 2.0 or something of this kind.
Okay, I ran PoshConsole, tested first clicks on tiles by commands issued manually and recorded a script: here it is, for your understanding what the structure of controls are (Inspect would show you similar results). In short, script does the following (I did the following):
– press Win to get to Start Screen
– click onto Photos, hovering near links to Photos, Skydrive photos, Facebook, Flickr, and text fields at the top of the screen
– click on Mail, select an account, hover between letters.
Below is the output:
Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | ` Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | ` Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | ` Get-UIAPane -AutomationId 'ScrollViewer' | ` Get-UIAPane -AutomationId 'content' | ` Get-UIAListItem -AutomationId 'microsoft.windowsphotos_8wekyb3d8bbwe!Microsoft.WindowsLive.ModernPhotos' -Name 'Photos LargeTile1 Image'; #InvokePattern #ValuePattern #ExpandCollapsePattern #GridItemPattern #SelectionItemPattern #ScrollItemPattern Get-UIAPane -AutomationId 'Start menu window' -Name 'Start menu'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAText -Name 'Launching Photos'; #ValuePattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAImage; #ValuePattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAText -Name 'SkyDrive photos'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIAListItem -Name 'Facebook photos See your albums'; #InvokePattern #SelectionItemPattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIACustom -Name 'Scrolling Container'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAText -Name 'Flickr photos' | ` #TextChangedEvent triggered #source title: of the type Document | ` Invoke-UIAListItemClick | ` #TextChangedEvent triggered #source title: of the type Document; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAText -Name 'Flickr photos'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIACustom -Name 'Scrolling Container' | ` #TextChangedEvent triggered #source title: of the type Document; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIAListItem -Name 'Facebook photos See your albums'; #InvokePattern #SelectionItemPattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIACustom -Name 'Scrolling Container'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAList | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAImage; #ValuePattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAText -Name 'Photos'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane -Name 'Windows Photos'; #ValuePattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAText -Name 'APP PREVIEW'; # no supported pattterns Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIAText -Name 'Photos'; # no supported pattterns Get-UIAWindow -AutomationId 'window' -Name 'PoSh - C:\1\poshconsole_f9bb2b127402\PoshConsole\bin\AnyCPU\Release (FileSystem)'; #WindowPattern #TransformPattern #SynchronizedInputPattern Get-UIAWindow -AutomationId 'window' -Name 'PoSh - C:\1\poshconsole_f9bb2b127402\PoshConsole\bin\AnyCPU\Release (FileSystem)' | ` Get-UIADocument | ` #TextChangedEvent triggered #source title: of the type Document; #TextPattern #SynchronizedInputPattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane -Name 'Windows Photos'; #ValuePattern Get-UIAWindow -Name 'Photos' | ` Get-UIAPane | ` Get-UIAPane -Name 'Windows Photos' | ` Get-UIACustom -Name 'Scrolling Container'; # no supported pattterns Get-UIAButton -AutomationId 'ImmersiveBackground' -Name 'Launch Start Menu' | ` #TextChangedEvent triggered #source title: of the type Document; #InvokePattern Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | ` Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | ` Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | ` Get-UIAPane -AutomationId 'ScrollViewer' | ` Get-UIAPane -AutomationId 'content' | ` Get-UIAListItem -AutomationId 'Microsoft.XboxLIVEGames_8wekyb3d8bbwe!Microsoft.XboxLIVEGames' -Name 'Xbox LIVE Games' | ` #TextChangedEvent triggered #source title: of the type Document; #InvokePattern #ValuePattern #ExpandCollapsePattern #GridItemPattern #SelectionItemPattern #ScrollItemPattern Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | ` Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | ` Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | ` Get-UIAPane -AutomationId 'ScrollViewer' | ` Get-UIAPane -AutomationId 'content' | ` Get-UIAListItem -AutomationId 'microsoft.windowscommunicationsapps_8wekyb3d8bbwe!Microsoft.WindowsLive.Mail' -Name 'Mail SwedishPod101.com, SwedishPod101 Word of the Day - 04/27/2012, Friday, April 27th, 2012 Your Swedish word of the day is: väg (Can you guess what it is? Click here'; #InvokePattern #ValuePattern #ExpandCollapsePattern #GridItemPattern #SelectionItemPattern #ScrollItemPattern Get-UIAWindow -Name 'Mail' | ` Get-UIAPane | ` Get-UIAPane -Name 'Mail' | ` Get-UIAList -Name 'Message List' | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAListItem -Name 'From Группа разработчиков службы Hotmail, Subject Совет для Hotmail: как избавиться от спама, Received April 18, 2012, Read' | ` Get-UIACustom -Name 'Группа разработчиков службы Hotmail Совет для Hotmail: как избавиться от спама 4/18/12 ********************************************************************** Windows Live Hotmail(r) ********************************************************************** Hotmail защищает от спама. Hotmail использует множество инструментов для фил From Группа разработчиков службы Hotmail, Subject Совет для Hotmail: как избавиться от спама, Received April 18, 2012, Read' | ` Get-UIAText -Name 'Совет для Hotmail: как избавиться от спама'; # no supported pattterns Get-UIAWindow -Name 'Mail' | ` Get-UIAPane | ` Get-UIAPane -Name 'Mail' | ` Get-UIAList -Name 'Message List' | ` Get-UIACustom -Name 'Scrolling Container' | ` Get-UIAListItem -Name 'From Пользователь denisebaker68@hotmail.com, Subject Пользователь denisebaker68@hotmail.com хочет стать вашим другом в службе Windows Live, Received April 17, 2012, Read' | ` Get-UIACustom -Name 'Пользователь denisebaker68@hotmail.com Пользователь denisebaker68@hotmail.com хочет стать вашим другом в службе Windows Live 4/17/12 Пользователь Пользователь denisebaker68@hotmail.com хочет добавить вас в свой список друзей в службе Windows Live <https://xxxxxxxxx From Пользователь denisebaker68@hotmail.com, Subject Пользователь denisebaker68@hotmail.com хочет стать вашим другом в службе Windows Live, Received April 17, 2012, Read' | ` Get-UIAText -Name 'Пользователь Пользователь denisebaker68@hotmail.com хочет добавить вас в свой список друзей в службе Windows Live <https://xxxxxxxxx' | ` #TextChangedEvent triggered #source title: of the type Document; # no supported pattterns Get-UIAWindow -Name 'Mail' | ` Get-UIAPane | ` Get-UIAPane -Name 'Mail' | ` Get-UIAMenuItem -Name 'Folders' | ` #TextChangedEvent triggered #source title: of the type Document | ` Invoke-UIAListItemClick | ` #TextChangedEvent triggered #source title: of the type Document | ` Invoke-UIAButtonClick | ` #TextChangedEvent triggered #source title: of the type Document | ` Invoke-UIAButtonClick | ` #TextChangedEvent triggered #source title: of the type Document; #InvokePattern
Beware: navigation among Metro items
To whom it might frighten: multi-string names. There are no AutomationId here, only complicated names to get a control.
The Start Screen is much better:
What’s inside? The Mail application:
Kobo:
As can be seen from the last two figures, UI Automation is available to some extent. Nevertheless, we are far away from the ultimate happiness: there’s almost nothing to do in Solitaire.
Dictionary.com is also automation-ready:
It’s time to please ourselves with one more fascinating pic from the Store and say to Microsoft: “Think more about automation engineers _before_ the release”:
Happy testing!
