Kinect Flash Tests

So this site has been down for a few days whilst I sorted out some issues with the hosting company, and a few emails have been lost/bounced – but everyhting should be sorted now. That’s also why this post is a couple of weeks late.

Anyway, a few weeks ago, I managed to free up some time to try and get our office’s shiny new Kinect working with Flash. We’d seen so many amazing videos of the things people were doing with it, that we thought we’d give it a go. it was a bit of a pain to do, but pretty fun to play with once it was all hooked up properly.

As far as I know, there’s currently 2 ways to go about getting your Kinect to work with Flash, both of which are community projects, as the official Microsoft drivers and SDK are yet to be released. So currently your two options are OpenKinect and OpenNi. I think both can work with Flash, however the following demos are all using OpenKinect. If you’re starting from scratch however, I’d suggest checking out OpenNi, as it allows access to the skeleton data of your players.

Here’s a brief video of some of the experiments I made –

Getting started –

Download and install the kinect drivers – for the camera, microphone and motor of the Kinect.
Download and install “Microsoft Visual C++ 2010 Redistributable Package”
Download and run “KinectAS3Server”
All available from the AS3Kinect page –
http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

If it doesn’t run, follow the comment of Shaun Husain –

I was having trouble getting the binary running on Vista, it was complaining that libusb0.dll wasn’t made for windows or something along those lines, coming up in an alert, to fix it I just downloaded libusbwin32 from here http://sourceforge.net/project… and pulled out the libusb_x86.dll and renamed to replace libusb0.dll in the as3kinect folder and it worked out fine. Thanks for making this happen Juan very cool stuff I’ve recently seen the demo of controlling XBMC so I threw together a Flex project to send the appropriate messages along to XBMC, now going to implement it using as3kinect.
Thanks again,
Shaun

Here’s the AS3-Server app, which has to be running for Kinect to communicate with Flash-

The AS3 Server package also contains a demo Flash file. The camera image received from the Kinect is on the right, and the depth information image on the left. You can set the depth which the Kinect is scanning and even control the tiliting motor on it from the Flash interface.


Things I’ve made.

Basic Kinect Setup –
A basic project template for accessing the Kinect depth data in a Flash application –

This file gets all items between 2 specified depths and works out their central point. (As shown by the red dot).

Mouse Emulator Loader –

This air app loads in any other flash swf file, and uses the Kinect depth data to fake the mouse position for the loaded swf. The idea was any mouse controlled game or app, could be loaded in and controlled with the Kinect. It sort of works, but not in the “all purpose” way intended – due to the handling of MouseEvents in Flash. As such, swfs to be loaded into this would have to be designed with these limitations in mind.

Flocking tests –

Items move around the screen based on the depth levels from the camera. Based on Soulwire’s Flocking Experiment.

Drum kit –

This file separates the depth information into 4 sectors. If anything is in the target depth range for that sector, it triggers a drum sample. All 4 sectors work independently, in an attempt to make a functioning drum kit.

Green screen –
An attempt to fake a green-screening effect by masking out the ordinary video feed from the camera based on the info from the depth camera. Unfortunately they don’t quite line up. The effect didn’t really work either.

Head tracking –

A basic 3D scene which rotates in 3D space based on where the user is. At the minute uses the same “central point of all objects in depth range” as the basis of the rotation. Switching to OpenNI, I could potentially get access to the user skeleton points and could use that to track the user’s head – so as they move around, the scene appears to move realistically, as if their monitor is a window looking into the 3d space.

Mona Lisa –


Her eyes really do follow you around the room!

Particles –

Based on UnitZeroOne’s brilliant particle tutorial.

Particles flow in line with the camera output and are coloured based on the Kinect’s depth sensor.

Notes –
The current version of the AS3-Server is an early build and is a bit unstable.
If you see this, close and restart the server.

Update –

Here’s a link to the source for these demos (25mb Zip) – http://dl.dropbox.com/u/5053444/AS3KinectSource.zip

13 thoughts on “Kinect Flash Tests”

  1. Yep, I’ll try and get them up in the next couple of days – just bear in mind, they’re very rough unoptimised demos :)

  2. Cheers Justin! I’ll hopefully have a video of the stuff I’ve been working on recently up soon too – background removal and skeleton based physics games!

  3. Hey Lawrie,

    Awesome job man. I’ve just started meddling around with the Kinect, and I had some errors with your source codes. Mainly with the got_depth function in the BasicKinect class for every project (Monalisa, BasicKinect, HeadTracking). CS5 kept giving error “1046: Type was not found or was not a compile-time constant: as3kinectWrapperEvent.”? The demo package you posted worked just fine though. Thanks!

  4. Hmm, I’ve not got the Kinect on this computer any more so it’s hard for me to check out. Is the Fla’s AS3 settings set up to look for classes in the “classes” folder?

  5. Can you post the Mouse Emulator Loader source file? Please!

    The AS3KinectSource.zip doesn’t include this one.

  6. Hi Carlos – I must have missed that one out of the Zip. Here’s a link to the files – http://dl.dropbox.com/u/5053444/FlashMouseLoader.zip

    I think I might have left it out, as I found out it couldn’t work like I’d hoped, so never updated it to the newer version of the AS3Kinect code – so it might not work out of the box, but should give you a good idea of how to go about doing something similar.

    Hope that helps.

  7. hey man im studying art and design in galway ireland, your stuff looks real cool, i have being collecting kinect apps and programs, and teaching myself processing and a bit of quartz. doing my best still its a slow process. My computer knowledge is growing. Still i downloaded you dropbox stuff opened the flash.app it show window still no kinect visuals, would love to mess with it, planing on making kinect movie, building rig at the moment for kinect and computer (mac 32 bit) Would be cool if you could tell me how to make this work. Cheers dude.

Leave a Reply

Your email address will not be published.