[Plugin] : Microphone record & compress

TL;DR : here's a plugin to access the microphone via Flash, encode the WAV in MP3 or OGG, and it gives you back a base64 string that represent the audio file.

<img src="https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/demo.png" border="0">

In the HTML5 draft, the browsers vendors are planning the use of the Microphone device.

On some systems and builds, you can even use it in conjunction with the webcam via getUserMedia(). However, there's no build anywhere to use the Local Stream Media to grab the audio from the microphone, and inject it inside an <audio> tag. Even if that code would exist, there's no realistic way to compress that audo in MP3 or OGG in Javascript for the moment.

However, we can hope that in two or three years from now, that the HTML5 spec would be advanced enough to allow that.

For the moment, here's a plugin that uses a Flash fallback to grab the Mic, and compress the WAV to MP3 and OGG.

Once that audio is recorded, you can insert it as a base64 string with the Base64 Audio plugin : http://www.scirra.com/forum/plugin-base64-audio_topic52192_post328265.html#328265.

The OGG encoder is modded to avoid a full memory clear and reload between each encoding, the MP3 encoder still need to reload itself between each encoding. That means the MP3 encoding part is usually faster than the OGG encoding on the first encoding, but it's slower on the following encoding.

I need to change the Shine encoder, however the change is heavier, because I need to have a proper Alchemy-encoding gcc installed, and it's not simple...

To do its job, the Flash part of the plugin display a Security Panel to allow the access to the microphone. That panel is shown and hidden when you need it. (On the HTML side, to hide the panel, I just set the width & height to 1 pixel, because a truly hidden div with a Flash inside won't execute any Flash if it's hidden). To avoid the "dead pixel" effect of that 1 pixel wide div when you want it hidden, you can set the color background of that div, to let it better blend inside your game.

Since I didn't have the time to add a check to get the proper name of the browser, you need to encode the MP3 or OGG accordingly to the browser brand (Firefox won't play a MP3, Chrome will play both, Opera won't play OGG, and so on...).

The encoding code is done in Flash, I used the Alchemy tool to accelerate some code to the maximum. It's not native speed, but it's way faster than "simple" Flash, or Javascript. The encoding is asynchronous, so you can do other things while it's done.

The code of the encoder is going to be published on GitHub, to let you be sure that the code can be trusted (I'll publish everything back from my holidays).

The MP3 encoder is the Shine MP3 encoder ported to Flash, and the OGG encoder is from there : http://labs.byhook.com/2011/02/15/ogg-vorbis-encoder-for-flash-alchemy-series-part-1/

The demo : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/index.html

The .zip : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/pode_microphone.1.0.zip

The .capx : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/MicrophoneC2Demo2.capx

Nobody tested it yet ? It's easier for me to erase bugs if people test the plugins on their own config...

interesting plugin, but can't think of a way yet how to use it in my games/ programs :)

Might have a look at it someday just to look how it is set up to learn from it ;)

Ubivis : you can, for example, let a player speak his name in front of the mic, and add it to his profile <img src="smileys/smiley2.gif" border="0" align="middle" />.

not really needed but a nice idea. :)

Might be interesting for me when someone develops a speech2text plugin, so you can command the game with your voice ;)

If you were able to extract frequencies, you could potentially create some singing game, or even on a simple level, by testing volume, have the user make noises that are converted into terrain, for a runner/cave flyer type game haha.

I'm very interested in this and will play around with it this week and get back with you. Thank you for your work!

Maybe a possibility would be something similar to the DS games that use microphone? Like in the DS Zelda, where you can blow in the mic and put out candles?

Does it work in mobile browsers that support flash?

JohnnySix : if I have the time, I'll add the frequencies separation in the Flash code (in JS, it would be too slow for the moment, without a native FFT).

Sheepy : in the next build, I'm going to add a "microphone level" property, to let people use the microphone like in Zelda, like you describe.

It should work in mobile browsers supporting Flash, but I think that means only Android now. Since I don't have a Android device, people need to test and tell me.

Pode that�s a cool feature ;] Mics could have many interesting uses in a mobile platform.

I�ll see if I can get flash to work on my phone

think it's possible to send the OGG/MP3 stream over TCP to make a sort of voice chat?

Fimbul : not for the moment, since you need to do an Ajax get/post to a TCP socket, and browser can't open them.

However, you can send the base64 string via WebSockets.

(I'm also planning to do something with Flash P2P).

I think this plugin has a great deal of potential!

I'm not sure I've downloaded any plugins yet, but I'll give it a try.

Pode can you use any type of mic?

When I saw the title, the idea that jumped in my head was a Mad Lib style game, before the game starts, the player records a death sound, a weapon sound, a jump sound, victory yell, greet the princess... etc. Then the game is playable with whatever sounds they made.

It would be hilarious, imagine people sharing their versions on Facebook.

— Val : any mic connected to the machine will be picked by the Flash code.