Kinect Flash Tests

So this site has been down for a few days whilst I sorted out some issues with the hosting company, and a few emails have been lost/bounced – but everyhting should be sorted now. That’s also why this post is a couple of weeks late.

Anyway, a few weeks ago, I managed to free up some time to try and get our office’s shiny new Kinect working with Flash. We’d seen so many amazing videos of the things people were doing with it, that we thought we’d give it a go. it was a bit of a pain to do, but pretty fun to play with once it was all hooked up properly.

As far as I know, there’s currently 2 ways to go about getting your Kinect to work with Flash, both of which are community projects, as the official Microsoft drivers and SDK are yet to be released. So currently your two options are OpenKinect and OpenNi. I think both can work with Flash, however the following demos are all using OpenKinect. If you’re starting from scratch however, I’d suggest checking out OpenNi, as it allows access to the skeleton data of your players.

Here’s a brief video of some of the experiments I made –

Getting started –

Download and install the kinect drivers – for the camera, microphone and motor of the Kinect.
Download and install “Microsoft Visual C++ 2010 Redistributable Package”
Download and run “KinectAS3Server”
All available from the AS3Kinect page –
http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

If it doesn’t run, follow the comment of Shaun Husain –

I was having trouble getting the binary running on Vista, it was complaining that libusb0.dll wasn’t made for windows or something along those lines, coming up in an alert, to fix it I just downloaded libusbwin32 from here http://sourceforge.net/project… and pulled out the libusb_x86.dll and renamed to replace libusb0.dll in the as3kinect folder and it worked out fine. Thanks for making this happen Juan very cool stuff I’ve recently seen the demo of controlling XBMC so I threw together a Flex project to send the appropriate messages along to XBMC, now going to implement it using as3kinect.
Thanks again,
Shaun

Here’s the AS3-Server app, which has to be running for Kinect to communicate with Flash-

The AS3 Server package also contains a demo Flash file. The camera image received from the Kinect is on the right, and the depth information image on the left. You can set the depth which the Kinect is scanning and even control the tiliting motor on it from the Flash interface.


Things I’ve made.

Basic Kinect Setup –
A basic project template for accessing the Kinect depth data in a Flash application –

This file gets all items between 2 specified depths and works out their central point. (As shown by the red dot).

Mouse Emulator Loader –

This air app loads in any other flash swf file, and uses the Kinect depth data to fake the mouse position for the loaded swf. The idea was any mouse controlled game or app, could be loaded in and controlled with the Kinect. It sort of works, but not in the “all purpose” way intended – due to the handling of MouseEvents in Flash. As such, swfs to be loaded into this would have to be designed with these limitations in mind.

Flocking tests –

Items move around the screen based on the depth levels from the camera. Based on Soulwire’s Flocking Experiment.

Drum kit –

This file separates the depth information into 4 sectors. If anything is in the target depth range for that sector, it triggers a drum sample. All 4 sectors work independently, in an attempt to make a functioning drum kit.

Green screen –
An attempt to fake a green-screening effect by masking out the ordinary video feed from the camera based on the info from the depth camera. Unfortunately they don’t quite line up. The effect didn’t really work either.

Head tracking –

A basic 3D scene which rotates in 3D space based on where the user is. At the minute uses the same “central point of all objects in depth range” as the basis of the rotation. Switching to OpenNI, I could potentially get access to the user skeleton points and could use that to track the user’s head – so as they move around, the scene appears to move realistically, as if their monitor is a window looking into the 3d space.

Mona Lisa –


Her eyes really do follow you around the room!

Particles –

Based on UnitZeroOne’s brilliant particle tutorial.

Particles flow in line with the camera output and are coloured based on the Kinect’s depth sensor.

Notes –
The current version of the AS3-Server is an early build and is a bit unstable.
If you see this, close and restart the server.

Update –

Here’s a link to the source for these demos (25mb Zip) – http://dl.dropbox.com/u/5053444/AS3KinectSource.zip

Arduino Sewing Machine Guitar Pedal

A few days ago I saw a sewing machine pedal in a charity shop, and wondered if I could turn it into a guitar effects pedal. Through a bit of re-wiring and few lines of Processing, it turns out you can. The hardest bit was getting the pedal apart!

The pedal is basically a spring loaded potentiometer, so I wired it to send the values, through an Arduino to a Processing App, and from there, to the virtual guitar effects software – Guitar Rig.

It’s not a bad start, but it might take me a while to make a full copy of one of these –

VIMEO LIKES

The other day I had a little time on my hands, so I decided to make a Vimeo video explorer.

From the beginning, Vimeo was created by filmmakers and video creators who wanted to share their creative work, along with intimate personal moments of their everyday life. As time went on, like-minded people came to the site and built a community of positive, encouraging individuals with a wide range of video interests. We hope that you feel inspired to show us both your creative side as well as your friendly side.

What is a ‘like’?

A ‘like’ is quick and friendly way to let a video creator know that you appreciate or like his or her video.

What is the point of liking a video?

‘Liking’ a video is simple way to let other users know that you enjoy their videos. Once you look around Vimeo, you’ll quickly start to find videos that you enjoy. By ‘liking’ a video you are helping to promote content to your contacts and subscribers as well as giving that video a creator a nice feeling.

So the Vimeo Explorer site looks up all the videos a user has “liked” – then displays their thumbnails in this semi-3D grid. You can then click a thumbnail to read the details of that video – and click again to watch the video.

There’s a few weird bugs in there still – sometimes the description doesn’t match the selected video, and the embedded vimeo player has a few odd quirks of its own. But on the whole, I was quite pleased with how it turned out.

As well as trying the Vimeo API, I got the chance to try out some cool effects, such as text to speech conversion and animated displacement maps.
See – http://wonderfl.net/c/rgyc and http://wonderfl.net/c/tLMY for source code demos.

The page is set to default to my personal Vimeo Likes – but by passing in any Vimeo user’s ID, you can load theirs.

For example – you can check out the likes of –
James Alliban of Tech Arts Vimeo Channel
Mario Klingemann aka Quasimondo
ideajockey.tv – a vlog about creative idea videos
Erik natzke – Flash based generative artist
John Davey of Flash On The Beach

Hopefully you’ll find it a fun way of exploring the videos liked by various people.
Cheers.
Lawrie.

New game – PuzzBox

I’ve finally got all the contract details sorted with one of my new games – PuzzBox, a stylish little Simon-esque puzzle game – sponsored by the guys over at BubbleDevil.

It’s a basic “copy the number pattern as it gets longer” memory puzzle game – but I’ve jazzed it up with some PV3D, particle effects and some displacement maps which give a nice glitchy effect.

Give it a play and let me know what you think.


Arduino Project + Home Office

So, I’ve been pretty busy lately – back to work, which is great – getting stuck back into interesting Flash development for an awesome client, with some really good people.

But that doesn’t mean I’m falling behind on my other on-going projects. I’ve still got 2 games almost finalised, and my interactive arts project with Josh Sadler is coming on well.

For the arts piece, I’ve recently picked up an Arduino.

What’s an Arduino?

Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It’s intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments.

Sweet! I’ve been interested in HCI, hardware hacking and alternate input devices for a while – so I really should have picked up an Arduino ages ago!

Check out this beaut –

Here’s the kit I bought – complete with LEDs, potentiometers, breadboard and LCD screen. There’s lots of possibilites here. On the day it arrived, Josh and I managed to get it hooked up to some code we’d written (in PureData and Processing respectively).

And as a bonus, just for mentioning it, I get extra added blog cred. Just check out this XKCD –

In other news, the design site Inspiredology recently ran part two of their round up of “INSPIRING WORK ENVIRONMENTS”.

It’s fun to notice how every single one is a mac! I guess mac users are just more stylish. My current “home office” certainly offers no competition –

Yeah, that’s right. I code on a stool.