## Kinect – OpenNI and NITE

After a long battle with various installer packages and drivers, I finally got OpenNI, NITE, and the Kinect working with Flash.
This means that I can now access the skeleton data from users, and make fun things like this –

On the whole, this guide is a good resource for getting everything setup correctly, although I did still have lots of issues.

## Kinect Flash Tests

So this site has been down for a few days whilst I sorted out some issues with the hosting company, and a few emails have been lost/bounced – but everyhting should be sorted now. That’s also why this post is a couple of weeks late.

Anyway, a few weeks ago, I managed to free up some time to try and get our office’s shiny new Kinect working with Flash. We’d seen so many amazing videos of the things people were doing with it, that we thought we’d give it a go. it was a bit of a pain to do, but pretty fun to play with once it was all hooked up properly.

As far as I know, there’s currently 2 ways to go about getting your Kinect to work with Flash, both of which are community projects, as the official Microsoft drivers and SDK are yet to be released. So currently your two options are OpenKinect and OpenNi. I think both can work with Flash, however the following demos are all using OpenKinect. If you’re starting from scratch however, I’d suggest checking out OpenNi, as it allows access to the skeleton data of your players.

Here’s a brief video of some of the experiments I made –

Getting started –

Download and install the kinect drivers – for the camera, microphone and motor of the Kinect.
All available from the AS3Kinect page –
http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

If it doesn’t run, follow the comment of Shaun Husain –

I was having trouble getting the binary running on Vista, it was complaining that libusb0.dll wasn’t made for windows or something along those lines, coming up in an alert, to fix it I just downloaded libusbwin32 from here http://sourceforge.net/project… and pulled out the libusb_x86.dll and renamed to replace libusb0.dll in the as3kinect folder and it worked out fine. Thanks for making this happen Juan very cool stuff I’ve recently seen the demo of controlling XBMC so I threw together a Flex project to send the appropriate messages along to XBMC, now going to implement it using as3kinect.
Thanks again,
Shaun

Here’s the AS3-Server app, which has to be running for Kinect to communicate with Flash-

The AS3 Server package also contains a demo Flash file. The camera image received from the Kinect is on the right, and the depth information image on the left. You can set the depth which the Kinect is scanning and even control the tiliting motor on it from the Flash interface.

Basic Kinect Setup –
A basic project template for accessing the Kinect depth data in a Flash application –

This file gets all items between 2 specified depths and works out their central point. (As shown by the red dot).

This air app loads in any other flash swf file, and uses the Kinect depth data to fake the mouse position for the loaded swf. The idea was any mouse controlled game or app, could be loaded in and controlled with the Kinect. It sort of works, but not in the “all purpose” way intended – due to the handling of MouseEvents in Flash. As such, swfs to be loaded into this would have to be designed with these limitations in mind.

Flocking tests –

Items move around the screen based on the depth levels from the camera. Based on Soulwire’s Flocking Experiment.

Drum kit –

This file separates the depth information into 4 sectors. If anything is in the target depth range for that sector, it triggers a drum sample. All 4 sectors work independently, in an attempt to make a functioning drum kit.

Green screen –
An attempt to fake a green-screening effect by masking out the ordinary video feed from the camera based on the info from the depth camera. Unfortunately they don’t quite line up. The effect didn’t really work either.

A basic 3D scene which rotates in 3D space based on where the user is. At the minute uses the same “central point of all objects in depth range” as the basis of the rotation. Switching to OpenNI, I could potentially get access to the user skeleton points and could use that to track the user’s head – so as they move around, the scene appears to move realistically, as if their monitor is a window looking into the 3d space.

Mona Lisa –

Her eyes really do follow you around the room!

Particles –

Based on UnitZeroOne’s brilliant particle tutorial.

Particles flow in line with the camera output and are coloured based on the Kinect’s depth sensor.

Notes –
The current version of the AS3-Server is an early build and is a bit unstable.
If you see this, close and restart the server.

## Update –

Here’s a link to the source for these demos (25mb Zip) – http://dl.dropbox.com/u/5053444/AS3KinectSource.zip

## UK Postcode Validation in AS3

Let me file this post under boring but useful. Recently I worked on a project where I had to validate UK postcodes in flash. The method I used in the end was a regular expression I found, with a bit of formatting logic added. This should work for all UK postcodes.

var validPostcode:String   =  "LS12ED";
var validPostcode2:String   = "LS1 2ED";
var invalidPostcode:String =  "NOTAPOSTCODE";

trace(validateUkPostcode(validPostcode));//true
trace(validateUkPostcode(validPostcode2));//true
trace(validateUkPostcode(invalidPostcode));//false

function validateUkPostcode(str:String):Boolean {

if(str.indexOf(" ")==-1){
var l:int = str.length;
str=str.substr(0,l-3)+" "+str.substr(l-3,l);
}

var pattern:RegExp =  /[A-Z]{1,2}[0-9R][0-9A-Z]? [0-9][ABD-HJLNP-UW-Z]{2}/i

var result:Object = pattern.exec(str);
if(result == null) {
return false;
} else {
return true;
}
}

## Mos Wanted

Hello!

First – the obligatory “I’ve not posted much for ages” bit. I’ve been working away on lots of interesting projects, including a couple of E-Learning applications, some games, a few more music experiments, more arduino tests and I’m starting to get underway on some iPhone and Android apps too – both native and Air based! So hopefully I’ll have some more updates in the near future. But today I want to talk about our fancy new charity fund raising site – Mos Wanted.

During November I will be growing a moustache to help spread awareness of Prostate Cancer and raise some cash for charity.

Movember challenges men to change their appearance and the face of men’s health by growing a moustache. The rules are simple, start Movember 1st clean shaven and then grow a moustache for the entire month. The moustache becomes the ribbon for men’s health, the means by which awareness and funds are raised for cancers that affect men. Much like the commitment to run or walk for charity, the men of Movember commit to growing a moustache for 30 days.

If you have a bit of spare change you wouldn’t mind donating to this great cause, then here’s a link to my sponsorship form – http://uk.movember.com/donate/your-details/member_id/526842, where you’ll get to see me sporting a huge mustache, as well as my current progress.

You can also check out the site I’ve made with some guys from work documenting our Mo-Growing progress –
http://moswanted.co.uk/

Or you can use the photobooth site we made –

Which lets you test drive mustaches – either from saved photos –

I worked with some of the great designers in my office, to make this site promoting our fund-raising efforts. All the work was done in our downtime, when no client work was on and over a few late nights. I’m pretty proud of the results, and I’ll try to post a code round up, some of the lessons learned and maybe even some code snippets when I get a chance.

Also – The site is today’s site of the day on DesignLicks and is in the running for an FWA – so please head over to the FWA site and vote for it!

Cheers guys.
With your help we might reach out £3000 target!
Lawrie.

## Cellular Automidi – Audio App

Presenting my latest Flash Music Toy App Thing –

## Cellular AutoMidi!

Cellular AutoMidi is a generative music app, making “music” based on a modified Cellular Automata algorithm.

It’s an AIR app –

Check it out in action in the video below, first using Flash dynamic sounds, then using Flash Midi Server to control a synth –

According to Wikipedia –

A cellular automaton is a discrete model studied in computability theory, mathematics, physics, complexity science, theoretical biology and microstructure modeling. It consists of a regular grid of cells, each in one of a finite number of states, such as “On” and “Off” . For each cell, a set of cells called its neighborhood is defined relative to the specified cell. An initial state (time t=0) is selected by assigning a state for each cell. A new generation is created (advancing t by 1), according to some fixed rule (generally, a mathematical function) that determines the new state of each cell in terms of the current state of the cell and the states of the cells in its neighborhood. For example, the rule might be that the cell is “On” in the next generation if exactly two of the cells in the neighborhood are “On” in the current generation, otherwise the cell is “Off” in the next generation. Typically, the rule for updating the state of cells is the same for each cell and does not change over time, and is applied to the whole grid simultaneously, though exceptions are known.

…..yep! Basically – each cell can be alive or dead. Once in a generation, each cell looks at it’s surrounding cells, and dies if it is lonely or overcrowded. If a dead cell has an optimum amount of neighbors, it will come to life! Each generation, all the cells which have come to life will sound a note. The notes are assigned based on the cell’s y position, and are all in the pentatonic scale.

There’s a few controls at the bottom which change how things work too.

• Start/Stop – Starts/Stops the automation.
• Export – Exports the current pattern to the clipboard. You can send it to friends, or save it for later, then load in with the load button.
• Clear down – Stop and clear the current pattern.
• Law Mode – An error when coding the cell rules gave this other odd mode.
• Skip Audio – Just show the cell animations.
• Sing Dead – Instead of singing the recently revived notes, sing for the recently deceased.
• Note duration – Alter the system speed.

Also, along the top there are banks of preset systems. Click play to start a saved pattern, and click assign to assign the pattern currently displayed to that button. You can also trigger each pattern with the keyboard keys 1-8.

When you press Export, your pattern is automatically copied to the clipboard, so you can save it, or share it with people. Here’s a pattern I made – you can load it by pasting it into the load box, and pressing Load!

1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,1 ,2 ,2 ,1 ,2 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,1 ,2 ,2 ,1 ,2 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1

HINT – When drawing patterns, symmetrical patterns seem to evolve nicely.

The app is fully compatible with Flash Midi Server (my Flash to MIDI audio interface app) – it checks to see if it is running when the app is launched. If it’s not, then it uses fancy Flash Player 10 dynamic audio! I’m hoping to roll this out to the Flash Midi Server class soon – so any app which tried to access the midi server and fails, will use the dynamic audio as a backup. I’ll keep you posted!

In case you missed it, you can download the app here – it weighs in at just 79kb including icons! The screenshot images above are 79k! If you don’t have Adobe Air installed, the link will prompt you to download that too.

I’d love to hear what you think of it – and see any patterns you come up with, so please post them in the comments.
Have fun!
Law.