Flash Midi Server – I/O version. Work in progress

A couple of days ago, a guy called Alfonso posted a comment on the Flash Midi Server page asking if the app could pass data from a midi controller to flash. FMS had always been designed to send out data – as I was originally interested in using Flash as the driving force for generative audio apps. But adding Midi input to the setup should open some more interesting possibilities.
I spent a couple of hours setting up an Input version last night – and got a demo version working.

In the image, the controller positions on the Novation Nocturn are mirrored in the swf.

I need to add the new input functions to the current FMS server app – but expect an Input / Output Flash Midi Server to come in the next few days.

SION AS3 Sound library instruments

Another quick – post. I’ve been playing with the great SION AS3 sound library lately, but couldn’t find a list of all the instruments available. So here they are – all the ones that worked for me at least –

var waveArray:Array = ["sine","saw","triangle8","triangle","square","noise","snoise","konami","ma1","beep","ramp","valsound.bass1","valsound.bell1","valsound.brass1","valsound.guitar1","valsound.lead1","valsound.percus1","valsound.piano1","valsound.se1","valsound.special1","valsound.strpad1","valsound.wind1","valsound.world1","midi.piano1","midi.chrom1","midi.organ1","midi.guitar1","midi.bass1","midi.strings1","midi.ensemble1","midi.brass1","midi.reed1","midi.pipe1","midi.lead1","midi.pad1","midi.fx1","midi.world1","midi.percus1","midi.se1","midi.drum24"];

More details on SION here – http://sites.google.com/site/sioncenter/

And here’s a great Wonderfl demo of what you can do with SION –

SiON SoundObject Quartet – wonderfl build flash online

Musical Scales as midi note numbers.

For a couple of audio projects, I needed some nice musical scales, in midi note format (0-127). I couldn’t find a good list online, so had to knock together a quick script to trace some out. Here’s some that I’m using in a current project, and also the AS source to find any other ones you may want.

Using this PDF as a guide – http://8.brm.sk/Scales_Chords.pdf – enter the scale name, and the notes which are active in that scale. For example, the first scale in the image below, the Adonai Malakh (Israel), we would set the code as –

//Enter scale name - 
var scaleName:String="AdonaiMalakh(Israel)";
//Enter the notes in each octave to use - see - http://8.brm.sk/Scales_Chords.pdf
var acceptedArray:Array = [0,1,2,3,5,7,9,10];

Which would give us the output –

var AdonaiMalakh(Israel):Array =[48,49,50,51,53,55,57,58,60,61,62,63,65,67,69,70,72,73,74,75,77,79,81,82,84,85,86,87,89,91,93,94];

//Enter scale name - 
var scaleName:String="Zirafkend";
//Enter the notes in each octave to use - see - http://8.brm.sk/Scales_Chords.pdf
var acceptedArray:Array = [0,2,3,5,7,8,9,11];

var octaveStep:int=0;
var finalArray:Array = [];

//You can limit the scales to any length - it will retrieve the middle section of the scale.
var limitArray:int = 32;

for(var i:int=0;i<12;i++){
	for(var j:int=0;j

And here's some scales. These are chopped to the middle 32 notes in each sequence.

var BiYu:Array =[24,27,31,34,36,39,43,46,48,51,55,58,60,63,67,70,72,75,79,82,84,87,91,94,96,99,103,106,108,111,115,118];
var Blues:Array =[41,42,43,46,48,51,53,54,55,58,60,63,65,66,67,70,72,75,77,78,79,82,84,87,89,90,91,94,96,99,101,102];
var BluesDiminished:Array =[48,49,51,52,54,55,56,58,60,61,63,64,66,67,68,70,72,73,75,76,78,79,80,82,84,85,87,88,90,91,92,94];
var Dorian:Array =[25,27,30,32,34,37,39,42,44,46,49,51,54,56,58,61,63,66,68,70,73,75,78,80,82,85,87,90,92,94,97,99];
var FullMinor:Array =[51,53,55,56,57,58,59,60,62,63,65,67,68,69,70,71,72,74,75,77,79,80,81,82,83,84,86,87,89,91,92,93];
var HarmonicMajor:Array =[44,47,48,50,52,53,55,56,59,60,62,64,65,67,68,71,72,74,76,77,79,80,83,84,86,88,89,91,92,95,96,98];
var Hawaiian:Array =[39,43,45,47,48,50,51,55,57,59,60,62,63,67,69,71,72,74,75,79,81,83,84,86,87,91,93,95,96,98,99,103];
var IonianSharp5:Array =[45,47,48,50,52,53,56,57,59,60,62,64,65,68,69,71,72,74,76,77,80,81,83,84,86,88,89,92,93,95,96,98];
var JazzMinor:Array =[45,47,48,50,51,53,55,57,59,60,62,63,65,67,69,71,72,74,75,77,79,81,83,84,86,87,89,91,93,95,96,98];
var Lydian:Array =[45,47,48,50,52,54,55,57,59,60,62,64,66,67,69,71,72,74,76,78,79,81,83,84,86,88,90,91,93,95,96,98];
var Major:Array =[43,45,48,50,51,52,54,55,57,60,62,63,64,66,67,69,72,74,75,76,78,79,81,84,86,87,88,90,91,93,96,98];
var Mixolydian:Array =[45,46,48,50,52,53,55,57,58,60,62,64,65,67,69,70,72,74,76,77,79,81,82,84,86,88,89,91,93,94,96,98];
var Oriental:Array =[45,46,48,49,52,53,54,57,58,60,61,64,65,66,69,70,72,73,76,77,78,81,82,84,85,88,89,90,93,94,96,97];
var SuperLocrian:Array =[44,46,48,49,51,52,54,56,58,60,61,63,64,66,68,70,72,73,75,76,78,80,82,84,85,87,88,90,92,94,96,97];
var VerdiEnigmaticAscending:Array =[46,47,48,49,52,54,56,58,59,60,61,64,66,68,70,71,72,73,76,78,80,82,83,84,85,88,90,92,94,95,96,97];
var Zirafkend:Array =[48,50,51,53,55,56,57,59,60,62,63,65,67,68,69,71,72,74,75,77,79,80,81,83,84,86,87,89,91,92,93,95];

I've also put the code up on Wonderfl - so you can use it straight from your browser.

Kinect Flash Tests

So this site has been down for a few days whilst I sorted out some issues with the hosting company, and a few emails have been lost/bounced – but everyhting should be sorted now. That’s also why this post is a couple of weeks late.

Anyway, a few weeks ago, I managed to free up some time to try and get our office’s shiny new Kinect working with Flash. We’d seen so many amazing videos of the things people were doing with it, that we thought we’d give it a go. it was a bit of a pain to do, but pretty fun to play with once it was all hooked up properly.

As far as I know, there’s currently 2 ways to go about getting your Kinect to work with Flash, both of which are community projects, as the official Microsoft drivers and SDK are yet to be released. So currently your two options are OpenKinect and OpenNi. I think both can work with Flash, however the following demos are all using OpenKinect. If you’re starting from scratch however, I’d suggest checking out OpenNi, as it allows access to the skeleton data of your players.

Here’s a brief video of some of the experiments I made –

Getting started –

Download and install the kinect drivers – for the camera, microphone and motor of the Kinect.
Download and install “Microsoft Visual C++ 2010 Redistributable Package”
Download and run “KinectAS3Server”
All available from the AS3Kinect page –
http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

If it doesn’t run, follow the comment of Shaun Husain –

I was having trouble getting the binary running on Vista, it was complaining that libusb0.dll wasn’t made for windows or something along those lines, coming up in an alert, to fix it I just downloaded libusbwin32 from here http://sourceforge.net/project… and pulled out the libusb_x86.dll and renamed to replace libusb0.dll in the as3kinect folder and it worked out fine. Thanks for making this happen Juan very cool stuff I’ve recently seen the demo of controlling XBMC so I threw together a Flex project to send the appropriate messages along to XBMC, now going to implement it using as3kinect.
Thanks again,
Shaun

Here’s the AS3-Server app, which has to be running for Kinect to communicate with Flash-

The AS3 Server package also contains a demo Flash file. The camera image received from the Kinect is on the right, and the depth information image on the left. You can set the depth which the Kinect is scanning and even control the tiliting motor on it from the Flash interface.


Things I’ve made.

Basic Kinect Setup –
A basic project template for accessing the Kinect depth data in a Flash application –

This file gets all items between 2 specified depths and works out their central point. (As shown by the red dot).

Mouse Emulator Loader –

This air app loads in any other flash swf file, and uses the Kinect depth data to fake the mouse position for the loaded swf. The idea was any mouse controlled game or app, could be loaded in and controlled with the Kinect. It sort of works, but not in the “all purpose” way intended – due to the handling of MouseEvents in Flash. As such, swfs to be loaded into this would have to be designed with these limitations in mind.

Flocking tests –

Items move around the screen based on the depth levels from the camera. Based on Soulwire’s Flocking Experiment.

Drum kit –

This file separates the depth information into 4 sectors. If anything is in the target depth range for that sector, it triggers a drum sample. All 4 sectors work independently, in an attempt to make a functioning drum kit.

Green screen –
An attempt to fake a green-screening effect by masking out the ordinary video feed from the camera based on the info from the depth camera. Unfortunately they don’t quite line up. The effect didn’t really work either.

Head tracking –

A basic 3D scene which rotates in 3D space based on where the user is. At the minute uses the same “central point of all objects in depth range” as the basis of the rotation. Switching to OpenNI, I could potentially get access to the user skeleton points and could use that to track the user’s head – so as they move around, the scene appears to move realistically, as if their monitor is a window looking into the 3d space.

Mona Lisa –


Her eyes really do follow you around the room!

Particles –

Based on UnitZeroOne’s brilliant particle tutorial.

Particles flow in line with the camera output and are coloured based on the Kinect’s depth sensor.

Notes –
The current version of the AS3-Server is an early build and is a bit unstable.
If you see this, close and restart the server.

Update –

Here’s a link to the source for these demos (25mb Zip) – http://dl.dropbox.com/u/5053444/AS3KinectSource.zip