Kinect Flash Tests

So this site has been down for a few days whilst I sorted out some issues with the hosting company, and a few emails have been lost/bounced – but everyhting should be sorted now. That’s also why this post is a couple of weeks late.

Anyway, a few weeks ago, I managed to free up some time to try and get our office’s shiny new Kinect working with Flash. We’d seen so many amazing videos of the things people were doing with it, that we thought we’d give it a go. it was a bit of a pain to do, but pretty fun to play with once it was all hooked up properly.

As far as I know, there’s currently 2 ways to go about getting your Kinect to work with Flash, both of which are community projects, as the official Microsoft drivers and SDK are yet to be released. So currently your two options are OpenKinect and OpenNi. I think both can work with Flash, however the following demos are all using OpenKinect. If you’re starting from scratch however, I’d suggest checking out OpenNi, as it allows access to the skeleton data of your players.

Here’s a brief video of some of the experiments I made –

Getting started –

Download and install the kinect drivers – for the camera, microphone and motor of the Kinect.
Download and install “Microsoft Visual C++ 2010 Redistributable Package”
Download and run “KinectAS3Server”
All available from the AS3Kinect page –
http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

If it doesn’t run, follow the comment of Shaun Husain –

I was having trouble getting the binary running on Vista, it was complaining that libusb0.dll wasn’t made for windows or something along those lines, coming up in an alert, to fix it I just downloaded libusbwin32 from here http://sourceforge.net/project… and pulled out the libusb_x86.dll and renamed to replace libusb0.dll in the as3kinect folder and it worked out fine. Thanks for making this happen Juan very cool stuff I’ve recently seen the demo of controlling XBMC so I threw together a Flex project to send the appropriate messages along to XBMC, now going to implement it using as3kinect.
Thanks again,
Shaun

Here’s the AS3-Server app, which has to be running for Kinect to communicate with Flash-

The AS3 Server package also contains a demo Flash file. The camera image received from the Kinect is on the right, and the depth information image on the left. You can set the depth which the Kinect is scanning and even control the tiliting motor on it from the Flash interface.


Things I’ve made.

Basic Kinect Setup –
A basic project template for accessing the Kinect depth data in a Flash application –

This file gets all items between 2 specified depths and works out their central point. (As shown by the red dot).

Mouse Emulator Loader –

This air app loads in any other flash swf file, and uses the Kinect depth data to fake the mouse position for the loaded swf. The idea was any mouse controlled game or app, could be loaded in and controlled with the Kinect. It sort of works, but not in the “all purpose” way intended – due to the handling of MouseEvents in Flash. As such, swfs to be loaded into this would have to be designed with these limitations in mind.

Flocking tests –

Items move around the screen based on the depth levels from the camera. Based on Soulwire’s Flocking Experiment.

Drum kit –

This file separates the depth information into 4 sectors. If anything is in the target depth range for that sector, it triggers a drum sample. All 4 sectors work independently, in an attempt to make a functioning drum kit.

Green screen –
An attempt to fake a green-screening effect by masking out the ordinary video feed from the camera based on the info from the depth camera. Unfortunately they don’t quite line up. The effect didn’t really work either.

Head tracking –

A basic 3D scene which rotates in 3D space based on where the user is. At the minute uses the same “central point of all objects in depth range” as the basis of the rotation. Switching to OpenNI, I could potentially get access to the user skeleton points and could use that to track the user’s head – so as they move around, the scene appears to move realistically, as if their monitor is a window looking into the 3d space.

Mona Lisa –


Her eyes really do follow you around the room!

Particles –

Based on UnitZeroOne’s brilliant particle tutorial.

Particles flow in line with the camera output and are coloured based on the Kinect’s depth sensor.

Notes –
The current version of the AS3-Server is an early build and is a bit unstable.
If you see this, close and restart the server.

Update –

Here’s a link to the source for these demos (25mb Zip) – http://dl.dropbox.com/u/5053444/AS3KinectSource.zip

UK Postcode Validation in AS3

Let me file this post under boring but useful. Recently I worked on a project where I had to validate UK postcodes in flash. The method I used in the end was a regular expression I found, with a bit of formatting logic added. This should work for all UK postcodes.

var validPostcode:String   =  "LS12ED";
var validPostcode2:String   = "LS1 2ED";
var invalidPostcode:String =  "NOTAPOSTCODE";

trace(validateUkPostcode(validPostcode));//true
trace(validateUkPostcode(validPostcode2));//true
trace(validateUkPostcode(invalidPostcode));//false

function validateUkPostcode(str:String):Boolean {
	
	if(str.indexOf(" ")==-1){
		trace("Adding postcode space");
		var l:int = str.length;
		str=str.substr(0,l-3)+" "+str.substr(l-3,l);
	}

	var pattern:RegExp =  /[A-Z]{1,2}[0-9R][0-9A-Z]? [0-9][ABD-HJLNP-UW-Z]{2}/i

	var result:Object = pattern.exec(str);
	if(result == null) {
		return false;
	} else {
		return true;
	}
}

Cellular Automidi – Audio App

Presenting my latest Flash Music Toy App Thing –

Cellular AutoMidi!

Cellular AutoMidi is a generative music app, making “music” based on a modified Cellular Automata algorithm.

It’s an AIR app –

YOU CAN DOWNLOAD IT HERE FOR FREE!


Check it out in action in the video below, first using Flash dynamic sounds, then using Flash Midi Server to control a synth –

Cellular AutoMidi – Generative Audio Flash AIR App from Lawrie Cape on Vimeo.

According to Wikipedia –

A cellular automaton is a discrete model studied in computability theory, mathematics, physics, complexity science, theoretical biology and microstructure modeling. It consists of a regular grid of cells, each in one of a finite number of states, such as “On” and “Off” . For each cell, a set of cells called its neighborhood is defined relative to the specified cell. An initial state (time t=0) is selected by assigning a state for each cell. A new generation is created (advancing t by 1), according to some fixed rule (generally, a mathematical function) that determines the new state of each cell in terms of the current state of the cell and the states of the cells in its neighborhood. For example, the rule might be that the cell is “On” in the next generation if exactly two of the cells in the neighborhood are “On” in the current generation, otherwise the cell is “Off” in the next generation. Typically, the rule for updating the state of cells is the same for each cell and does not change over time, and is applied to the whole grid simultaneously, though exceptions are known.

…..yep! Basically – each cell can be alive or dead. Once in a generation, each cell looks at it’s surrounding cells, and dies if it is lonely or overcrowded. If a dead cell has an optimum amount of neighbors, it will come to life! Each generation, all the cells which have come to life will sound a note. The notes are assigned based on the cell’s y position, and are all in the pentatonic scale.

There’s a few controls at the bottom which change how things work too.

  • Start/Stop – Starts/Stops the automation.
  • Load – Loads a pattern from the text box.
  • Export – Exports the current pattern to the clipboard. You can send it to friends, or save it for later, then load in with the load button.
  • Clear down – Stop and clear the current pattern.
  • Law Mode – An error when coding the cell rules gave this other odd mode.
  • Skip Audio – Just show the cell animations.
  • Sing Dead – Instead of singing the recently revived notes, sing for the recently deceased.
  • Note duration – Alter the system speed.

Also, along the top there are banks of preset systems. Click play to start a saved pattern, and click assign to assign the pattern currently displayed to that button. You can also trigger each pattern with the keyboard keys 1-8.

When you press Export, your pattern is automatically copied to the clipboard, so you can save it, or share it with people. Here’s a pattern I made – you can load it by pasting it into the load box, and pressing Load!

1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,1 ,2 ,2 ,1 ,2 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,1 ,2 ,2 ,1 ,2 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,2 ,2 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,2 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1 ,1

HINT – When drawing patterns, symmetrical patterns seem to evolve nicely.

The app is fully compatible with Flash Midi Server (my Flash to MIDI audio interface app) – it checks to see if it is running when the app is launched. If it’s not, then it uses fancy Flash Player 10 dynamic audio! I’m hoping to roll this out to the Flash Midi Server class soon – so any app which tried to access the midi server and fails, will use the dynamic audio as a backup. I’ll keep you posted!

In case you missed it, you can download the app here – it weighs in at just 79kb including icons! The screenshot images above are 79k! If you don’t have Adobe Air installed, the link will prompt you to download that too.

DOWNLOAD IT HERE!

I’d love to hear what you think of it – and see any patterns you come up with, so please post them in the comments.
Have fun!
Law.

–By downloading the app, you are agreeing to the licence below —

Copyright (c) 2010, Lawrie Cape

All rights reserved.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Arduino Sewing Machine Guitar Pedal

A few days ago I saw a sewing machine pedal in a charity shop, and wondered if I could turn it into a guitar effects pedal. Through a bit of re-wiring and few lines of Processing, it turns out you can. The hardest bit was getting the pedal apart!

The pedal is basically a spring loaded potentiometer, so I wired it to send the values, through an Arduino to a Processing App, and from there, to the virtual guitar effects software – Guitar Rig.

It’s not a bad start, but it might take me a while to make a full copy of one of these –

VIMEO LIKES

The other day I had a little time on my hands, so I decided to make a Vimeo video explorer.

From the beginning, Vimeo was created by filmmakers and video creators who wanted to share their creative work, along with intimate personal moments of their everyday life. As time went on, like-minded people came to the site and built a community of positive, encouraging individuals with a wide range of video interests. We hope that you feel inspired to show us both your creative side as well as your friendly side.

What is a ‘like’?

A ‘like’ is quick and friendly way to let a video creator know that you appreciate or like his or her video.

What is the point of liking a video?

‘Liking’ a video is simple way to let other users know that you enjoy their videos. Once you look around Vimeo, you’ll quickly start to find videos that you enjoy. By ‘liking’ a video you are helping to promote content to your contacts and subscribers as well as giving that video a creator a nice feeling.

So the Vimeo Explorer site looks up all the videos a user has “liked” – then displays their thumbnails in this semi-3D grid. You can then click a thumbnail to read the details of that video – and click again to watch the video.

There’s a few weird bugs in there still – sometimes the description doesn’t match the selected video, and the embedded vimeo player has a few odd quirks of its own. But on the whole, I was quite pleased with how it turned out.

As well as trying the Vimeo API, I got the chance to try out some cool effects, such as text to speech conversion and animated displacement maps.
See – http://wonderfl.net/c/rgyc and http://wonderfl.net/c/tLMY for source code demos.

The page is set to default to my personal Vimeo Likes – but by passing in any Vimeo user’s ID, you can load theirs.

For example – you can check out the likes of –
James Alliban of Tech Arts Vimeo Channel
Mario Klingemann aka Quasimondo
ideajockey.tv – a vlog about creative idea videos
Erik natzke – Flash based generative artist
John Davey of Flash On The Beach

Hopefully you’ll find it a fun way of exploring the videos liked by various people.
Cheers.
Lawrie.