Using an iOS device to control a game on your browser

Accelerometer support is available in Mobile Safari and this enables the browser to sense movement, speed and direction with Javascript on iPhone and iPad devices. If we push the movement data from the iOS device instantly through a nodejs server to a browser, then we can control a game on the browser. This is exactly what we have put together in this fun little html5 demo.

Try the game before you go through the components that make up this demo.

iPhone / iPad controller

The user scans the QR code or visits the unique link created for them from the demo landing page to start the controller. The controller code reads accelerometer values and then passes the movement to the nodejs server.

Key parts of the iOS device controller code

//Detect if the browser supports DeviceMotionEvent
if (window.DeviceMotionEvent != undefined) {
//ondevicemotion is fired when iOS device detects motion
  window.ondevicemotion = function(e) {
//ax is the movement on the x axis.
//This motion is used to move the ship in the game
  ax = event.accelerationIncludingGravity.x * 5;
  ay = event.accelerationIncludingGravity.y * 5;

//Status 0 is start, 1 is left, 2 is right, 3 is stay
if(status == 0){ //initial condition
  status = 3; //stay
  socket.emit('spaceChange', {'ax': 3});
  statusmsg = 'Waiting for movement';
}
if(ax > 14 && status != 2){ //move right on device
  status = 2;
  socket.emit('spaceChange', {'ax': 2});
  statusmsg = 'Moving ship right';
}
if(ax < -14 && status != 1){ //move left on device      status = 1;      socket.emit('spaceChange', {'ax': 1});			         statusmsg = 'Moving ship left';  }  if(ax > -14 && ax < 14 && status != 3){ //device held steady
  status = 3;
  socket.emit('spaceChange', {'ax': 3});
  statusmsg = 'Ship held steady';
}

The controller code detects movement from accelerometer and only pushes data to the nodejs server if there is a change in direction of the device. This reduces the data pushed from the phone to the server when compared to sending  all the live accelerometer values to server. Finally, the value 14 in ax is set as a threshold to detect if the user is moving the device to the left or right.

Node.js server with SocketIO (Server side)

Node.js is a server-side JavaScript environment based on Google’s V8 Javascript engine. Programs are written in JavaScript, using event-driven, asynchronous I/O to minimize overhead and maximize scalability. Socket.IO sits on top of Node.js and makes it easy to deliver realtime data between almost every browser and the node server. Socket.IO code runs on the server and the client side. This makes Node.js and Socket.IO ideal candidates to be able push accelerometer values instantly from mobile safari to the desktop browser.

Each visitor to the game landing page is assigned an unique random id that links the browser and the iOS safari instance. User loads the URL provided on their iOS device and then safari detects accelerometer motion, which is then pushed by socket.io on client side to the node server. The randomly generated id is used as a room for the user so that push from user device is only sent to a particular browser.

Server side Socket.IO code

//Start on connection
io.sockets.on('connection', function (socket) {

 //Set room for user when connection is made
 socket.on('setChannel', function (data) {
 socket.join(data.channelName);
 });

 //When iOS device moves send data to browser
 //Multiple browsers can be listening to same device
 socket.on('spaceChange', function (data) {
 socket.broadcast.to(data.channelName)
                   .emit("spaceChanges",data);
 });

});

Browser game and client side Socket.IO

The browser game can be played with the keyboard arrows or with iOS device movement. The space invaders game itself behaves like a standard HTML5 game which uses HTML canvas. Thanks to Erop Balyshev for developing the well tested game in a couple of days. The html game code is commented and you should be able to understand the underlying code for the game quite easily. The socket communication and movement based on feedback from the iOS device was later plugged into the game code.

Code below shows how data from server is handled on the browser.

//iOS detection and corresponding action
var lastkey = 37;
var dataStart=0;
  socket.on('connect', function() {
   //if sockets gets disconnected then mention room again
   socket.emit('setChannel',
              {'channelName': '<!--?php echo $randRoom; ?-->'});
  });

  socket.on('spaceChanges', function (data) {
     if(dataStart == 0){
        //First movement data arrived
        document.getElementById('status').innerHTML
                  = 'Receiving data from your iOS device';
        dataStart = 1;
     }
    ax = data.ax;
    var	posob=new Object();
    if(ax == 2){
        //move right
        lastkey = 39;
    	posob.keyCode = lastkey;
        posob.type = 'keydown';
        document.getElementById('status').innerHTML
                   = 'iOS device tilted right';
    }
    if(ax == 1){
        //move left
        lastkey = 37;
    	posob.keyCode = lastkey;
        posob.type = 'keydown';
        document.getElementById('status').innerHTML
                   = 'iOS device tilted left';
    }
    if(ax == 3){
        //hold ship in place
    	posob.keyCode = lastkey;
	posob.type = 'keyup';
        document.getElementById('status').innerHTML
                   = 'iOS device held steady';
    }
    //Send action received above
    keypressaction(posob);
  });

//Fire automatically once first data starts
window.setInterval(function(){
  if(dataStart == 1){
    var posob=new Object();
    posob.keyCode = 32;
    posob.type = 'keydown';
    keypressaction(posob);
    posob.keyCode = 32;
    posob.type = 'keyup';
    keypressaction(posob);
  }
//Timer is correctly a shot ever 200ms.
//Decrease 200 to lower for even faster firing!
}, 200);

Conclusions

- We are very impressed with the instant movement detection and control over the game.
- You can also run multiple browsers on your desktop with the same id get parameter and have it all controlled with the same iOS controller.
- Current space invaders game uses only X axis movement on the iOS device. However, all three dimension values are available on iOS safari ondevicemotion method. It is technically possible to do much more with the iOS device. Probably the ship can fire when user shakes the device?

UPDATE: Watch a presentation of the HTML5 space game by Roshan Abraham

Enjoyed the demo? Looking forward to your feedback and ideas.

Also read...

Comments

  1. Kuzman Tancred said on :

    Cool demo!

    I didn’t realise that you can grab accelerometer values from Safari on an iOS device.

  2. Pingback: jsc: space invaders « The book of inspiration

  3. Jack said on :

    This is a great idea, and definitely has more applications than Space Invaders. One thing that would bother me, though, is the responsiveness of the controls. Because the input and the display aren’t connected locally, there’s an unavoidable gap between sending a command from the iPhone and displaying it in the browser. This is disorienting, and if you imagine something like a 3d camera moving around in a 3d space, that gap creates motion sickness, even if it’s only ~50ms.

    The biggest problem with this is that there’s no way to fake the display to improve perceived responsiveness! You can’t display something immediately while waiting for the server to respond, reducing the perceived input gap, because displaying something relies on the server responding.

  4. Jay Vaughan said on :

    Jack, I believe its an interesting GUI problem, but the thing is any lag at all can be appropriately indicated to the user, somehow. When the server doesn’t know, it just needs to say it doesn’t know (or, at the very least, achieve a GUI sync across the device/game interface, such that the user interface gui (lets call it a HUD), and the web-based game interface are at least time-sync’ed. This will make it easier to dissolve the lag with eye candy.

  5. Tatonka said on :

    I believe the felt gap in responsiveness has a lot to do with the choice of the accelerometer as input. To demo the actual “mobile-device to server to client remote control web socket thingy” .. maybe a more simple (and potentially faster) button interface would have been better.

    For the time being, I wouldn’t worry about motion sickness in 3D spaces, since this is rather far off for a browser implementation anyways. Lets grab the low hanging fruit here.

    Very cool tech demo .. definitively something I will try to utilize. Thanks.

    T.

  6. Colin Myerscough said on :

    This is the best I have seen in years!! I had to quit playing before I got addicted to the sweetness of the used technoligy.

    Keep up the good work!!

  7. Pingback: Blog ALL » Control this desktop Space Invaders game with your iPhone

  8. Pingback: Control this desktop Space Invaders game with your iPhone – - Tech News AggregatorTech News Aggregator

  9. Pingback: Toki Solutions » Broadcast & Technology Solutions Partner » Control this desktop Space Invaders game with your iPhone

  10. Paul D. Waite said on :

    It’s a bit over my head, so apologies if I’m misunderstanding things, but is there any scope for using Web Sockets (which I think has some support on desktop and mobile Safari, see caniuse.com/#search=websockets) to take the server out of the equation?

  11. php-manual said on :

    @Paul, Socketio is setup to fall back from websockets, xhr-polling, flash sockets and so on. So websockets need not be available on the browser.

    You can take server out of the equation but that would need you to be on the same LAN probably? Also default out the box socketio cannot be used.

  12. Pingback: Accelerometer-based game control using an iOS device courtesy of HTML5 - Hack a Day

  13. Pingback: Hackaday « Cool Internet Projects

  14. Philip said on :

    I can play it with no problems with my Desire Z running Android version 4.0.3 (something based off Cyanogenmod)

    right and left are switched, but else it works great. (meaning, i have to lean the device left to make the thingie go right and vice-versa)

  15. Dina said on :

    All Android 2.3 users, you can use Opera Mobile to play this game ;)

  16. Blaanid said on :

    I’m pretty sure the demos for Firefox Mobile included a demo showing it implements HTML5, including a 2.5D diorama that moved as you tilted the phone. All client side stuff, I think.

  17. RyanBruke said on :

    mobile dev nub here. What else can Safari pull from the device? Seems to be many interesting applications for interactivity.

  18. Pingback: Use Your iOS Device to Control Browser Based Games | ChurchMag

  19. somebody said on :

    please add option to invert controls in this demo so that android users could play it normaly to

    • php-manual said on :

      @somebody Thanks for the suggestion, we had few other users ask the same suggestion for Andorid based off Cyanogenmod. We will have a look at what we can do. Can you let us know the exact OS that you are using?

      As a temporary solution you can hold your phone facing the monitor and you should be able to play the game normally :)

  20. Dietmar said on :

    Hello! Great idea! Is it possible to get the sources of that demo?
    Would like to try it in my LAN and have some ideas what can be done apart from playing with that – even though playing is cool….:-)

  21. Pingback: Accelerometer-based game control using an iOS device courtesy of HTML5 | CisforComputers

  22. Steve Longhurst said on :

    It’s all working , a very cool bit of tech it is too.

  23. Pingback: Spaceship Pilot, CARPE Slider, CasperJS, lazyload

  24. kamend said on :

    huy guys, I have been playing around with Node.js,Socket.io and Mobile Safari recently. I am just curious are you aware of the problem that when Safari goes to sleep (user presses the Home button) the socket will disconnect after a while and when you open Safari again it will crash. Do you know any solutions for this problem?

  25. Moritz U. said on :

    This is awesome, and it also works on the Galaxy Nexus running Android 4.0.4. Only the accelerator values seems switched so the ship runs to the right when I tilt my device to the left.

  26. Jenny said on :

    Thanks. The demo works really well.

Comments are closed.