Rendering the audio captured by a Windows Phone device

This small app displays the audio captured by the microphone of a Windows Phone device and displays it as a continous waveform on the screen using the XNA framework.

A slightly modified version of the app (changing color when touching the screen) can be found in the Windows Phone Marketplace.

Background

In order to capture the audio on a Windows Phone device, you need an instance to the default microphone (Microphone.Default), decide how often you want samples using the BufferDuration-property and hook up the BufferReady-event. Then you control the capturing with the Start() and Stop() methods.

The microphone is giving you samples at a fixed rate of 16 000 Hz, i.e. 16 000 samples per second. There is a property SampleRate that will tell this value. This means that you won’t be able to capture audio of higher frequency than 8000 Hz (without distortion) according to the sampling theorem.

You are also limited when it comes to choose the value for the BufferDuration-property; it must be between 0.1 and 1 seconds (100 – 1000 ms) in 10ms-steps. This means that you must choose a value of 100, 110, 120, …, 990, 1000 milliseconds.

When the microphone event BufferReady is fired, you should call the microphone.GetData(myBuffer)-method, in order to copy the samples from the microphone’s internal buffer to a buffer that belongs to you. The recorded audio comes in the form of a byte-array, but since the samples are actually signed 16-bits integers (i.e. an integer in the range of -32’768 … 32’767), you will probably need to do some convertion before you can process them.

How this application works

The way this application works is keeping a fixed number of narrow images, here called “(image) slices”, arranged in a linked list. The images are rendered on the screen and smoothly moved from the right to the left. When the left-most slice has gone off the screen, it is moved to the far right (still outside the screen) in order to create the illusion of an unlimited number of images.

Each slice holds the rendered samples from the content of one microphone buffer. When the buffer is filled by the microphone mechanism, the rightmost slice (outside of the screen) is rendered with these new samples and started to be moved inwards the screen.

The speed of how fast the slices are moving across the screen is correlated to the duration of the buffer in such a way that the slices are moved a total of “one slice width” during the time the microphone is capturing the next buffer.

Since the buffer of captured audio is rendered as graphic on a texture as soon it is received, there is no reason to keep any old buffer data. Therefore the application only keeps one buffer in memory which is reused over and over.

A flag is set each time the microphone buffer is ready. Since the BufferReady event is fired on the main thread, there is no need for any lock-mechanism.

In the Update()-method of the XNA app, the flag is checked whether new data has arrived, and if so, the slice in line is drawn. In the Draw()-method, the slices are drawn on the screen and slightly moved as time goes by.

The complete Visual Studio solution file can be downloaded from here.

Here’s a description of the structure of the main “Game”-class.

Some constants:

Fields regarding the microphone and the captured data:

Choose a color that is almost transparent (the last of the four parameters; it’s the red, green, blue and alpha-component of the color). The reason is that many samples are drawn on top of each other, and keeping each individual sample almost see-through makes an interesting visual effect.

The drawing classes. The white pixel texture is doing all the drawing.

The size of each image slice.

There’s no need to keep a reference to the linked list itself; just the first and last link. These links keeps references to their neighbors. The currentImageSlice is the one to draw on the next time.

The speed of the slices moving across the screen.

In order to know how far the current samples should be moved, the application must keep track of when they appeared.

 The signal that tells the Update()-method that there is new data to handle.

 The density of samples per pixel.

Here’s the constructor. In it the graphics mode is set and the microphone is wired up and asked to start listening.

In the XNA’s LoadContent nothing is actually loaded since the app is not dependent on any predrawn images. The SpriteBatch is created, the white pixel texture is generated and the image slices are initialized (as black images).

The CreateSliceImages is calculating how many slices that are needed to cover the entire screen (plus two so there’s room for movement). In the end of the method the regular RenderSamples-method is called in order to initial all the images. Since there is no data yet (all samples are zero) it will generate black images.

The XNA’s UnloadContent is just cleaning up what the LoadContent created.

The event handler to the microphone’s BufferReady-event. It copies the data from the microphone buffer and raises the flag that new data has arrived.

The XNA’s Update method checks the phone’s Back-button to see if it’s time to quit. After that it checks the flag to see if new data has been recorded. If so, the new samples are rendered by calling the RenderSamles-method.

The XNA’s Draw-method takes care of drawing the rendered slices. It handles the two screen orientation modes; landscape and portrait, by scaling the images accordingly. If it is landscape mode the height of the images are squeezed and if it is portrait mode the width of the images are squeezed.

When all is setup, the method iterates through the images and render them one-by-one on the screen, adjusted a bit along the X-axis to make up for the time that has passed.

The RenderSamples is taking a RenderTarget2D as an argument, which is the texture to be drawn on. The routine iterates through the samples and render them one by one.

Badgers Rock

Windows Phone TileBadgers Rock is a small, completely free (no ads) Windows Phone game I created using the XNA framework during the autumn of 2011.
You can find it on the Windows Phone Marketplace and on Facebook.The latest released version (V1.4) has both an online high score and cool retro sound.There was a contest arranged by the “Windows Phone Sweden” Facebook page, that ended on February 3, 2012, and the Winner Niklas Dahlman successfully completed all thirty levels with “three stars” on each levels. Not an easy task considering some levels are really hard and time-demanding to complete. He won a Nokia Bluetooth Stereo Headset BH-111. Congratulations!

The online highscore is provided and hosted by http://mogade.com/. It is free, open source and have a very easy API to integrate. You can choose to host your data yourself or let them do it for you. Really nice.

All the sounds in the game were taken from the http://www.freesound.org/. Here is a complete list of the sounds that I used in the game.

Unholy characters in code

Just a head’s up — do try it at home — it’s most likely that your co-workers will not approve of these worst-practice naming standards…!

Prolog

It’s generally only when it comes to unit test method names that I allow myself to use underscores. Since those names tend to (and should) be a little chatty, it’s nice to separate the words to increase readability. That’s the first reason. The other reason is to be able to distinguish between Pascal-cased [variable] names in the sentence, and pure English words. For example; if you don’t use underscores it’s hard[er] to understand whether the name

means “Checking whether my name is empty” or “Checking whether MyName is empty”. Utilizing underscores between the actual words would clear that out.

Now to some funny business. For the faint hearted, please stop reading now!

Exotic characters

Well, underscores are nice enough, but you would go crazy having to read a whole book with them between each word, right? So, giving the fact that modern languages like C# now support foreign characters in addition to the English alphabet, and the fact that the number of charcters in total is rather large, maybe there are some obscure characters for us to use? And as luck would have it; there are. A couple, in fact.

First of all, of course not all unicode characters may be used in a name, just as little as e.g. a #-character would do. But there are a couple of characters that actually appears as spaces on screen, but are valid in type and method names. The character U+1160 (aka the Hangul Jungseong Filler) is one of those. Check this out:

Although (or since) this doesn’t generate any compiler errors, other developers stumbling across this piece of code might get pretty puzzled…

Please note: your source files will need to be saved in full unicode.

If you think that this is too hard to digest, and insist on having a character you can actually see, you might want to try these babies out:

Taking it even worse

If you really want to mess with your co-developers mind, you can even use a character with zero width (i.e. totally invisible). You will actually only notice it when you step it over using the arrow keys; then you’ll feel as the key press “didn’t take” since passing the character doesn’t move the caret.

You can for instance use these weird characters to make it appear as if you are “reusing” the same variable name many times by inserting one or more of these characters in the variable name. One of these invisible characters are U+200B (aka Zero Width Space). In the following example, the zero width space character is inserted 0, 1 and up to 4 times between “my” and “Value” in the different variable declarations. Please note that the code is perfectly legal and will declare five different variables (virtually appearing the same).

Quite nasty as they appear the same — but are in fact — totally different names… I guess the only way to make use of this character is in practical jokes…

The full monty

If you decide — against all common sense — that you really want to go for these unholy characters; you will find it pretty difficult since you’ll need to copy/paste them the whole time. Well, my friend, there’s a cure even for that. Just download and install The Microsoft Keyboard Layout Creator, and you’re able to assign one or more of these characters to physical keys of your keyboard.

Upon launch, go into the File menu and select “Load Existing Keyboard…”.

Load Existing Keyboard...

This will bring up a listbox with all the currently installed keyboard layouts on your machine. Select the one you are using.

Clicking the key-buttons on the main window brings up the small dialog where you can remap the selected key:

Press the “All…”-button to see the full dialog with all the modifier options (+shift/ctrl/alt etc):

Please be aware that there is a restriction of chaning the spacebar key to a non-space character. I tried to map the combination shift+ctrl+space to the “Hangul Jungseong Filler” (the invisible character) but was denied.


Maybe for the best! Tongue out I had to settle with ctrl+shift + the underscore key, which works good enough.

When you’re done with your keyboard tweaking, go into the Project menu and select “Build DLL and Setup Package”. This action will create a keyboard setup file which you can install on your PC.

After installing the package you can enjoy your new keyboard layout!

Wrapping it up

So, the real use of these exotic characters are maybe just to play some practical joke on someone. Or, one might construct an obfuscator utilizing these characters: imagine renaming all your names into different combinations of the Hangul Jungseong Filler and the Zero Width Space characters. Plain invisible code, my friend! Mohahahaa!