RoomWare – Using the phone as a remote control

Posted on November 5, 2009


[Addition: Feb 25, 2010] At the FITC Amsterdam I saw a “phone as remore control” demoed by Infrared5. Today Jeff Winder published an article where he used RemoteDroid to achieve the same results. Basic principle of this approach: either the “server” or the “client” connects to the other via the local (WiFi). Once it makes contact, server and client establish a peer-to-peer connection. Infrared automated the process by pinging the IP-numbers in the network. The latency Infrared5 found is about 20 milliseconds.

[November 11, 2009] In the past 20 days I hacked together the parts that shape part of the RoomWare / Phone interactions.

Demo and explenation (a silent movie)

Brief overview of the parts

It comprises of the following parts:

  1. A (Android) phone client with
    1. A bluetooth scanner returning found bluetooth devices with MAC address and name
    2. A http connector to the RoomWare Identity Broker
    3. A socket client to connect to a RoomWare server
  2. A database with registered bluetooth beacons
  3. A flash client using a socket connection to the RoomWare server

Design goals

The Mobile Phone is a very important component in the design goals for the first release of the new RoomWare components.

These design goals are have one single basis.

To make as simple and transparant as possible to:

  1. Set up your own and independent RoomWare environment
  2. Interface with your RoomWare powered environment.
  3. Build your own RoomWare Applications, using different technologies and devices

Read more about the other strategies here.

Ways to connect using a phone

Regarding the mobile phone, there are currently three ways to connect:

  1. By using an application installed on your phone or accessed via a web browser
  2. By dialing into a number and use the keypad from your phone.
  3. By using a QR-code and a web page

The dial-in variant is already developed by Valerie Ivangorodski and optimized for fast response October 2009.

Ready made components for use and download (2010)

  1. A gratis RoomWare Phone Client
  2. An Identity Broker (with an underlying database) in which you can register Bluetooth Beacons and hyperlinks to your Services
  3. A RoomWare Socket Server
    1. Already installed on a web server for direct use, already running on a web server
    2. Downloadable so you can set it up on your own machine
  4. Open Source code to build your own:
    1. RoomWare Socket Server
    2. RoomWare Socket Server Connectors for Flash and .NET – to build and connect your own RoomWare applications

Setup of the RoomWare phone application

Picture 101

1: Scanning for beacons

A beacon can any bluetooth dongle, from your laptop, phone or in the shape of a USB stick. All bluetooth dongles have a unique MAC address you can read with any Bluetooth scanner.

When you store that MAC address in a database, you can attach extra data to it, like a Service Name and a list of RoomWare Services connected to the RoomWare installation that dongle represent.

When the applications scans for beacons, it polls the online Identity Broker database for each bluetooth MAC address it finds. When the MAC address matches a beacon MAC address in the database, it returns a Service Name and a Service Description to the phone.

2: Presenting the list of Remote Controls associated to the service

When you select a service, the phone will load the list of RoomWare Services connected to that Beacon.

There are two types of Services:

  1. Web pages
  2. Remote Controls.

A web page can be read-only contain specific information, like the name of a building and backgrounds about the past and present.

A Remote Control can be either:

  1. A HTML page with specific JavaScript calls to the RoomWare phone application
  2. A list-representation generated inside the RoomWare phone application

3: Connecting to RoomWare using the Android client

The Android client connects to a RoomWare server via a Socket connection. The Java Socket client I wrote consists of two parts: A socketListenerThread and a SocketHandler. The SocketHandler deals with creating and destroying the Socket and takes the raw data from the SocketListenerThread to pass it to the object that instantiated the SocketHandler.

To continously read the Sockets without blocking the rest of the application I used a timer that triggers the SocketListener every N milliseconds.

The RoomWare Remote Control XML

With the RoomWare Phone application you can connect to any RoomWare Server available online. To do this you need three basic parts:

  1. The host: The web-location of your RoomWare Server. You can run your own RoomWare server anywhere you like, by running the executable on a computer that is accessible via the web or via your network.
  2. The application key: The name of your RoomWare application cluster. You can run multiple clusters of RoomWare applications via the same server, making the environment scalable to any level you wish.
  3. The subscribe to list: The events you want to receive in this Remote Control client

This is what your XML can look like:

<?xml version="1.0" encoding="UTF-8"?> <remotecontrol>   <roomwareserver>     <host></host>     <applicationkey>roomware application 23</applicationkey>     <subscribeto>remotecontrolactions</subscribeto>   </roomwareserver>   <button>     <label>Turn lights on</label>     <instructionset>set light on</instructionset>     <broadcastto>lightswitch 2</broadcastto>   </button></remotecontrol>

Each Remote Control item is presented as a button. Each button has a label, a set of instructions and a “broadcast to” telling the RoomWare Socket Server to who the instructions should be sent.

The RoomWare application connector – an example

On the other side, waiting, are one or more RoomWare applications. In the video embedded above this is a Flash application containing a presentation. The basic code to connect and get data is quite simple, as all the hard work is done by the RoomWare Connector classes.

Code for a Flash RoomWare application, connecting to and receiving data from the RoomWare Server:

// Get the RoomWare communicator and message classesimport roomware.RoomWareCommunicator;import* // connection definition and message

var RWCD:RoomWareConnectionDefinition=new RoomWareConnectionDefinition();

// Define the connection settings for the RoomWare serverRWCD.subscribeTo="remotecontrol"RWCD.applicationKey="roomware application 23";RWCD.IP="";RWCD.port=8000;

var RWC:RoomWareCommunicator=new RoomWareCommunicator(RWCD,onData)

function onData(RWM:RoomWareMessage){  // Any kind of data, set by the sender of the message  // We agreed that the phone-application sends XML  var RWdata:XML=XML(RWM.body);  // Instructions are embedded in the RWdata XML  var instructions:String = RWdata.instructions;


Storing the Remote Control XML on your server

For the RoomWare Phone Client to be able to find your Remote Control, you need to put it on a place where the Phone Client can find it. Online. On which web-locations makes no difference.

Telling the Identity Broker where to find your Remote Control

The Identity Broker has the following tables:

  1. Beacons: This table contains the ID, name and description of your bleutooth beacon. You put it there by logging into the RoomWare Identity Broker and then create a new beacon. The ID is the MAC address of your bluetooth beacon.
  2. BeaconServices: This table is connected to the Beacons table and contains the names and hyperlinks to your roomWare services.

What is the role of the RoomWare Identity Broker?

The RoomWare Identity Broker serves the following purpose:

To offer a centralized platform where you can:

  1. Publish your public RoomWare Services
  2. Grant other people easy access to your public RoomWare Services
  3. State your social identities (like your Flickr name, your Linked-In profile, etcetera)
  4. Grant RoomWare services easy access to your social identities
  5. Shield your social identities on basis of trust: only those locations you trust can have access to your social identities

Where do your run your RoomWare applications?

You can install and run your RoomWare applications anywhere you like.

Release: the first quarter of 2010

As there is a lot of work still to do regarding the Identity Broker and the Mobile Application it will be released early 2010.

This is the ToDo list for December and Januari:

  1. ToDo for the mobile client
    1. Add a JavaScript interface to the Remote Control / Service browser so people can build Remote Controls for their phones using HTML or Flash Light and JavaScript –  This allows more freedom in the design of your mobile applicaitons
    2. Add a “Register this bluetooth device” functionality to the RoomWare mobile phone application, so that you can register your services directly from the found bluetooth devices
    3. Add a “bookmark this location” functionality for later reference
    4. Add an identifier to the requests the client sends based on the bluetooth ID of the phone (see the ToDo for local RoomWare installations “handshake” later)
  2. ToDo for the Identity Broker
    1. Create an encrypted login mechanism and a shielded environment where peoples identities are protected from identity skimmers
    2. Finalize the web interface
  3. ToDo for local RoomWare installations
    1. Create a local bluetooth scanner for Windows and Mac to perform a handshake with bluetooth devices and offer that data to the local application
    2. Create a handshake-protocol using the founhd bluetooth devices to assure that the person communicating with the RoomWare application is actually there