Digital Windows – DIY Passive Telepresence At Home

Long distances apart can make it tough to keep up friendships, and while the Internet has helped with this through the advent of e-mail, instant messaging and video conferencing like Skype none of these mediums can accurately duplicate the experience of being able to quickly ask and see “What’s going on?” to a friend or colleague across great distance.

E-mail is a bit too asynchronous. Instant messaging suffers the same problem, along with the lack of visual feedback and a frequently too small text window. Skype calls go the other direction, they’re slightly too synchronous: it’s not possible to automate them, and they use more bandwidth than something more laid back really needs to be successful. What’s missing is a passive solution to let people passively – and pervasively – interact with others with video feedback.

Enter the “Digital Window”. Originally conceived a year or two ago, the idea is simple: a panel mounted in a visible location connected to a camera which shows a passive viewing portal into another space via a similar device located elsewhere. Glance out the window into a previously authenticated friend or colleague’s space and see what they’re up to, and start a conversation if it’s interesting. Time not spent in front of the computer is often when some of the most interesting projects take place and if nobody knows they’re happening, it’s impossible to collaborate and seek feedback.

The idea became reality with some minor revisions when a software developer friend recently got married – what better gift than writing a video conferencing appliance on dedicated hardware using nothing but off-the-shelf components? I collaborated with another member of our group across the country to make the system a reality, and we bootstrapped it from the ground up using it to help continue its own development once some basics were in place.

The wedding gift, our implementation of the digital window, consists of the client-side appliances: two Asus netbooks running Windows XP, configured to automatically begin broadcasting via their internal webcams using Windows Media Encoder 9, and load the Viewer application in a web browser. The viewer application itself is a web page hosted on the streaming server itself, a simple automatically-refreshing container loading a set of borderless Windows Media Player controls to display the streams. The viewer code is self-refreshing and requires no maintenance.

Windows Media Encoder 9 was somewhat difficult to find as it’s no longer officially supported, but it’s still available from Softpedia. Its replacement is the Windows Expressions Encoder, a commercial product with a free version, but it’s a heavier package and the Asus netbooks don’t need to waste cycles running a bigger application than necessary.

When the system first launches, it automatically launches a Visual Basic script via a shortcut in the Startup folder. Through some trial and error, we learned we needed to set the shortcut to launch the script minimized to avoid popping up a command prompt window over the output. The script then sets up an Internet Explorer window, the Viewer application, and launches Windows Media Encoder. The script can be found below, with some identifying information removed for security reasons – you’ll need to fill in your own URLs to use this code if you’re going to clone our setup yourself.

On Error Resume next 

Set WshShell = WScript.CreateObject("WScript.Shell")
Set objIE = CreateObject("InternetExplorer.Application")

Set sHTMLTitle = "Viewer Application"

Return = WshShell.Run("Custom.wme") 'Set this to the location of the .WME profile you created in Windows Media Encoder
WSCript.Sleep(60000) 'Sleep to let the encoder load - it does a lot of stuff on start-up.

objIE.StatusBar = 1
objIE.Toolbar = 1
objIE.Width = 1024
objIE.Height = 600
objIE.Left = 0
objIE.Top = 0
objIE.Visible = 1
objIE.Navigate "Viewer Application URL"

' Wait for IE to finish loading before continuing.
Do While objIE.Busy: Loop
Do While objIE.ReadyState <> 4: Loop

WScript.Sleep(5000)

Set EncoderAgent = CreateObject("WMEncAgt.WMEncoderAgent", "127.0.0.1")  ' Grab the running encoder via the WMEncoderAgent.
Set Encoder = EncoderAgent.GetEncoder(EncoderAgent.EncoderNamesCollection.Item(0)) 

Encoder.Start ' Begin Streaming

WshShell.AppActivate("Viewer Application") 'Activate different possible titles of the window.
WshShell.AppActivate("Internet Explorer") 'Depends on the local system's configuration what IE is titled...covers possibilities.
WshShell.AppActivate("Windows Internet Explorer")

While 1=1  'Watchdog
Select Case Encoder.RunState
Case 5 '"STOPPED" State
WScript.Sleep 2500
Encoder.Start
WScript.Sleep 2500
WshShell.AppActivate("Viewer Application")
WshShell.AppActivate("Internet Explorer")
WshShell.AppActivate("Windows Internet Explorer")
WshShell.AppActivate("Site URL Goes Here in Titlebar")
WshShell.AppActivate("Site URL Goes Here in Titlebar - Viewer Application")
WScript.Sleep 100
End Select
Err.Clear
WScript.Sleep 1000
Wend

The script isn’t incredibly complicated. We’ve settled on using AppActivate commands because the programmatic pressing of the “Start Encoding” button in the watchdog loop causes Windows Media Encoder to pop to the foreground over the viewer, blocking the window – not what we want to happen. The AppActivate pops the viewer back to the foreground where it belongs

To make the system more robust and appliance-like, the system checks to see whether the encoder is active and if it’s not (due to a network drop or other connection error) to reconnect. The Asus netbooks have a webcam driver that likes to act up after a few days running, so we’ve configured them to reboot automatically at around 3AM every day. They’re configured to come online fully automatically, so this restart is seamless.

The result came out better than expected: by simply monitoring the encoder’s state and disabling popup error messages, the system is robust enough to overcome non-critical driver errors, client-side network drops, and even reconnect automatically if the server is taken offline due to connectivity or maintenance. It’s expandable, too: viewer can be transparently updated with new features (as it’s a web browser, after all) and new clients can be added limited only by your screen’s area. It’s also possible to create different viewer interfaces depending on the device – a flat panel monitor, a large TV or a netbook. The initial startup page could become a landing page, redirecting the viewer clients to pages based on their HTTP request, User-Agent, IP, or another mechanism.

The first time the system is configured, you must create a new Session in Windows Media Encoder and save it to a file on your hard drive. The Session contains the source, destination, security and formatting information for the stream. We’re streaming video only right now, although audio streaming may come in a later version. Unfortunately, a profile must be created for each system manually, as it contains information about the specific driver and device that will be used.

The Encoder session is configured in Push mode – connecting to the server directly, instead of the server connecting to the encoder. This allows the appliance to work from behind nearly all firewalls, and is an additional security feature as the encoder controlling the camera cannot be accessed from outside the network, so it’s not possible for an unauthorized party to tap into the video feed. Additional security (authentication and IP range) is provided on the server.

Finally, we’re streaming 320×240 at 4 FPS, ~100Kb/s. CBR was chosen due to its simplicity, as it’s less taxing on the Celeron and Atom CPUs in the netbooks – and although being labeled as constant, when the camera is pointed at an unchanging scene without much color the bandwidth really does drop down, for example if the camera is pointed at a wall, it’s shutter is closed, etc. This might seem low, but it’s designed to be highly portable and we had to keep screen resolution concerns in mind. The quality is quite acceptable for casual use as we’ve designed it.

We started off with four windows: one for each of us designers, one for the new couple to keep, and one for them to give away to someone else they’d like to bring into our shared sphere of passive video collaboration. More can be added limited only by bandwidth. The viewer application is quite simple, just some HTML and JavaScript to glue it all together. It’s based loosely on the reference code for an embedded Media Player provided by the server:

<html><head>
<META HTTP-EQUIV="PRAGMA" CONTENT="NO-CACHE">
<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=UTF-8">

<title>Viewer Application</title>
<style>	BODY	{
		font-size : medium;
		font-family : Tahoma, Arial, Sans Serif;
		font-weight : normal;
		font-style : normal;
		color : #FFFFFF;
		background-color : #000000;
		border-style : none;	}
	A	{
                color : #66CCFF;
		font-weight : bold;	}
</style>
<script type="text/javascript">
function refresh() {
  location.reload(true)
}

window.setInterval("refresh()",600000);

</script>
</head>
<body bgcolor="#FFFFFF" topmargin="0" leftmargin="0">
<table border="0" width="100%" height="80%" align="center">			

	<tr>
			<td width="100%">
			<P ALIGN="center">
				<OBJECT CLASSID="clsid:6BF52A52-394A-11D3-B153-00C04F79FAA6" ID="WMP" WIDTH="320" HEIGHT="240" >
					<PARAM NAME="Name" VALUE="WMP1">
					<PARAM NAME="uiMode" VALUE="none">
					<PARAM NAME="URL" VALUE="REDACTED1">
				</OBJECT>
				<OBJECT CLASSID="clsid:6BF52A52-394A-11D3-B153-00C04F79FAA6" ID="WMP" WIDTH="320" HEIGHT="240">
					<PARAM NAME="Name" VALUE="WMP2">
					<PARAM NAME="uiMode" VALUE="none">
					<PARAM NAME="URL" VALUE="REDACTED2">
				</OBJECT>
				<P>
			</td>
	</tr>
	<tr>
			<td width="100%">
			<P ALIGN="center">
				<OBJECT CLASSID="clsid:6BF52A52-394A-11D3-B153-00C04F79FAA6" ID="WMP" WIDTH="320" HEIGHT="240">
					<PARAM NAME="Name" VALUE="WMP3">
					<PARAM NAME="uiMode" VALUE="none">
					<PARAM NAME="URL" VALUE="REDACTED3">
				</OBJECT>
				<OBJECT CLASSID="clsid:6BF52A52-394A-11D3-B153-00C04F79FAA6" ID="WMP" WIDTH="320" HEIGHT="240">
					<PARAM NAME="Name" VALUE="WMP4">
					<PARAM NAME="uiMode" VALUE="none">
					<PARAM NAME="URL" VALUE="REDACTED4">
				</OBJECT>
			</P>
			</td>
	</tr>
	<tr>
		<td width="100%">
			<P align="center">
				<A HREF="Viewer.html">Refresh</A> (10 Minute Auto)
			</p>
		</TD>
	</TR>

</table>

</body>
</html>

It’s pretty simple and can be expanded or modified as needed.

On the server side, the viewer code and stream brokering is handled by Streaming Media Services running on Windows Server 2008 R2. The streaming is hosted on a business-class connection with a 5Mb/s upstream, which is more than sufficient for current and future applications: the incoming bandwidth scales linearly with the number of clients, and while the outgoing bandwidth scales as the square of the connected clients this issue is mitigated by controlling the encoder settings to use only approximately 100Kb/s per stream. Our four-encoder system uses only 400Kb/s downstream bandwidth and 1.6Mb/s upstream – it will scale transparently to around 12 clients before becoming bottlenecked on the connection, or around 8 clients with higher resolution and audio.

Advertisements
This entry was posted in Internet, Projects, Technology and tagged , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s