Tag Archives: Windows 8 CP

Metro automation: the demo is published


The demo is published within the files of version 0.7.4 (though it’s almost 0.7.5).

The picture taken from a Windows Server 8 Beta host through the RDP connection:

Advertisements

Metro automation: the first script recorded!


With a bit of free time, I built a PowerShell host that works over Start Screen and recorded my first Metro script. As a host I used PoshConsole (a flamboyant shell, and I couldn’t use autocomplete for the UIAutomation cmdlets) with the manifest. At first, I tried to compile BGShell, but it falls each run, perhaps as a matter of using Windows Forms Aero for .NET 2.0 or something of this kind.

Okay, I ran PoshConsole, tested first clicks on tiles by commands issued manually and recorded a script: here it is, for your understanding what the structure of controls are (Inspect would show you similar results). In short, script does the following (I did the following):

– press Win to get to Start Screen

– click onto Photos, hovering near links to Photos, Skydrive photos, Facebook, Flickr, and text fields at the top of the screen

– click on Mail, select an account, hover between letters.

Below is the output:

Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | `
	Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | `
	Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | `
	Get-UIAPane -AutomationId 'ScrollViewer' | `
	Get-UIAPane -AutomationId 'content' | `
	Get-UIAListItem -AutomationId 'microsoft.windowsphotos_8wekyb3d8bbwe!Microsoft.WindowsLive.ModernPhotos' -Name 'Photos LargeTile1 Image';
		#InvokePattern
		#ValuePattern
		#ExpandCollapsePattern
		#GridItemPattern
		#SelectionItemPattern
		#ScrollItemPattern

Get-UIAPane -AutomationId 'Start menu window' -Name 'Start menu';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAText -Name 'Launching Photos';
		#ValuePattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAImage;
		#ValuePattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAText -Name 'SkyDrive photos';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIAListItem -Name 'Facebook photos

See your albums';
		#InvokePattern
		#SelectionItemPattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIACustom -Name 'Scrolling Container';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAText -Name 'Flickr photos' | `
	#TextChangedEvent triggered
#source title:  of the type Document | `
	Invoke-UIAListItemClick | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAText -Name 'Flickr photos';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIACustom -Name 'Scrolling Container' | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIAListItem -Name 'Facebook photos

See your albums';
		#InvokePattern
		#SelectionItemPattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIACustom -Name 'Scrolling Container';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAList | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAImage;
		#ValuePattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAText -Name 'Photos';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane -Name 'Windows Photos';
		#ValuePattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAText -Name 'APP PREVIEW';
		# no supported pattterns

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIAText -Name 'Photos';
		# no supported pattterns

Get-UIAWindow -AutomationId 'window' -Name 'PoSh - C:\1\poshconsole_f9bb2b127402\PoshConsole\bin\AnyCPU\Release (FileSystem)';
		#WindowPattern
		#TransformPattern
		#SynchronizedInputPattern

Get-UIAWindow -AutomationId 'window' -Name 'PoSh - C:\1\poshconsole_f9bb2b127402\PoshConsole\bin\AnyCPU\Release (FileSystem)' | `
	Get-UIADocument | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		#TextPattern
		#SynchronizedInputPattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane -Name 'Windows Photos';
		#ValuePattern

Get-UIAWindow -Name 'Photos' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Windows Photos' | `
	Get-UIACustom -Name 'Scrolling Container';
		# no supported pattterns

Get-UIAButton -AutomationId 'ImmersiveBackground' -Name 'Launch Start Menu' | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		#InvokePattern

Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | `
	Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | `
	Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | `
	Get-UIAPane -AutomationId 'ScrollViewer' | `
	Get-UIAPane -AutomationId 'content' | `
	Get-UIAListItem -AutomationId 'Microsoft.XboxLIVEGames_8wekyb3d8bbwe!Microsoft.XboxLIVEGames' -Name 'Xbox LIVE Games' | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		#InvokePattern
		#ValuePattern
		#ExpandCollapsePattern
		#GridItemPattern
		#SelectionItemPattern
		#ScrollItemPattern

Get-UIAWindow -AutomationId 'Start menu window' -Name 'Start menu' | `
	Get-UIAPane -AutomationId 'Root' -Name 'Start menu' | `
	Get-UIAList -AutomationId 'GridRoot' -Name 'Start Apps' | `
	Get-UIAPane -AutomationId 'ScrollViewer' | `
	Get-UIAPane -AutomationId 'content' | `
	Get-UIAListItem -AutomationId 'microsoft.windowscommunicationsapps_8wekyb3d8bbwe!Microsoft.WindowsLive.Mail' -Name 'Mail SwedishPod101.com, SwedishPod101 Word of the Day - 04/27/2012, Friday, April 27th, 2012 Your Swedish word of the day is: väg (Can you guess what it is? Click here';
		#InvokePattern
		#ValuePattern
		#ExpandCollapsePattern
		#GridItemPattern
		#SelectionItemPattern
		#ScrollItemPattern

Get-UIAWindow -Name 'Mail' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Mail' | `
	Get-UIAList -Name 'Message List' | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAListItem -Name 'From Группа разработчиков службы Hotmail, Subject Совет для Hotmail: как избавиться от спама, Received ‎April ‎18, ‎2012, Read' | `
	Get-UIACustom -Name 'Группа разработчиков службы Hotmail

Совет для Hotmail: как избавиться от спама

‎4/‎18/‎12

********************************************************************** Windows Live Hotmail(r) ********************************************************************** Hotmail защищает от спама. Hotmail использует множество инструментов для фил

From Группа разработчиков службы Hotmail, Subject Совет для Hotmail: как избавиться от спама, Received ‎April ‎18, ‎2012, Read' | `
	Get-UIAText -Name 'Совет для Hotmail: как избавиться от спама';
		# no supported pattterns

Get-UIAWindow -Name 'Mail' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Mail' | `
	Get-UIAList -Name 'Message List' | `
	Get-UIACustom -Name 'Scrolling Container' | `
	Get-UIAListItem -Name 'From Пользователь denisebaker68@hotmail.com, Subject Пользователь denisebaker68@hotmail.com хочет стать вашим другом в службе Windows Live, Received ‎April ‎17, ‎2012, Read' | `
	Get-UIACustom -Name 'Пользователь denisebaker68@hotmail.com

Пользователь denisebaker68@hotmail.com  хочет стать вашим другом в службе Windows Live

‎4/‎17/‎12

Пользователь Пользователь denisebaker68@hotmail.com хочет добавить вас в свой список друзей в службе Windows Live <https://xxxxxxxxx

From Пользователь denisebaker68@hotmail.com, Subject Пользователь denisebaker68@hotmail.com хочет стать вашим другом в службе Windows Live, Received ‎April ‎17, ‎2012, Read' | `
	Get-UIAText -Name 'Пользователь Пользователь denisebaker68@hotmail.com хочет добавить вас в свой список друзей в службе Windows Live <https://xxxxxxxxx' | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		# no supported pattterns

Get-UIAWindow -Name 'Mail' | `
	Get-UIAPane | `
	Get-UIAPane -Name 'Mail' | `
	Get-UIAMenuItem -Name 'Folders' | `
	#TextChangedEvent triggered
#source title:  of the type Document | `
	Invoke-UIAListItemClick | `
	#TextChangedEvent triggered
#source title:  of the type Document | `
	Invoke-UIAButtonClick | `
	#TextChangedEvent triggered
#source title:  of the type Document | `
	Invoke-UIAButtonClick | `
	#TextChangedEvent triggered
#source title:  of the type Document;
		#InvokePattern

Beware: navigation among Metro items


To whom it might frighten: multi-string names. There are no AutomationId here, only complicated names to get a control.

The Start Screen is much better:

What’s inside? The Mail application:

Kobo:

As can be seen from the last two figures, UI Automation is available to some extent. Nevertheless, we are far away from the ultimate happiness: there’s almost nothing to do in Solitaire.

Dictionary.com is also automation-ready:

It’s time to please ourselves with one more fascinating pic from the Store and say to Microsoft: “Think more about automation engineers _before_ the release”:

Happy testing!

%d bloggers like this: