naku penda piya, naku taka Piya

...mpenziwe

A light-stepping,
gravity-defying,
time-traveling,
super-modeling,
puppet racing,
spaceship-flying,
grave-robbing,
effect-cheezing,
tux-rocking,
child-prodigy,
crazy-dancing,
dirty-rocking

legend.

From a Queen Song to a Better Music Search Engine

My recent ICASSP paper described a Dynamic Texture Model that can capture both the timbral AND temporal qualities of music. An article about this and how it contributes to our music search project by solving the "Bohemian Rhapsody Problem" just got posted on the Jacob's School of Engineering Site, UCSD's site and EurekAlert.org!

Read the article here and check out how the system can automatically divide Bohemian Rhapsody into homogenous segments and then we can tag each segment individually, rather than treating the whole song as a single unit:

Herd It in the news

Phenomenal reception to the Beta release of my music annotation game Herd It!

First we got written up in Voice of San Diego and that led to a news spot on Channel 6 (unfortunately, I was in Taiwan at ICASSP so Gert had to present!)

Then Robyn Williams interviewed Gert and I on the Science Show, resulting in loads of new users (at some times, too many for our server!).

Play Herd It.

Harmonograph

Ever since reading the funny little book, "Harmonograph: A Visual Guide to the Mathematics of Music", I've been rather intrigued by the Harmonograph. A harmonograph is a music visualization tool that was first devised in Victorian times. It consists of two (or more) pendulums which oscillate with different frequencies and are both connected to a single pen or marker. By tuning the frequencies to various harmonic intervals (octaves, fifths, thirds, etc.), the pattern drawn out by the pen traces various intricately beautiful geometric shapes. Back in ye olde days, well-heeled toffs used to set up their harmonographs at dinner parties and everyone would get together and marvel at the trippy patterns it would create. Presumably they were all bombed on opium at the time...

Anyway, after reading the book, I realized that it would be reasonably simple to build my own "virtual" harmonograph. I started some Actionscript code to implement the equations of motion for the pen:
$P_x = sin(\omega_x t)$ and $P_y = sin(\omega_y t)$
where $\omega_x$ is the frequency of oscillation in the x-direction, corresponding to the first note in the interval. Thus, to plot the interval between A4 and A5, we set $\omega_x =$ 440Hz and $\omega_y =$ 880Hz. The resulting pattern looks like this:

The next step was to add the sound of the interval that is being plotted - this is something that the Victorian gadabouts couldn't do and I thought that it would really help to get some nice synesthesia going on. Enter Flash 10 and the new sound synthesis capabilities!

So, invite over your dandy friends, bust out the opium pipe and enjoy the virtual harmonograph!

Herd It on Amazon Web Services

I've spent the past month figuring out all the details and complications of hosting my Facebook music annotation game. Herd It, on Amazon Web Services. We figured out that one of the reasons why the game didn't work well when more than 5 people connected was lack of bandwidth on our server to serve the Flash components. Also, we want to be ready for when Herd It becomes the next Desktop Tower Defense so AWS seemed like a great solution. It was an omen that, on the day that I was debating whether or not to bother to try to figure it all out, a senior manager from AWS gave a talk at UCSD...

Anyway, now that I've figured it all out, I think that AWS is great. However, there are a lot of hurdles to overcome in getting it to work so hopefully this will benefit someone (maybe even me, when I forget what I did).

Step 1 - Register for an AWS account.
If you've ever bought anything Amazon, this is as simple as adding AWS to your existing account. In particular, you will need 2 of their (many) web services:
EC2 (elastic compute cloud) - this is the "cloud" of servers that does all the processing.
S3 (simple storage service) - this is the storage bucket where you will keep all your data.

Step 2 - Figure out EC2.
EC2 works as follows:
You create an AMI - an "Amazon machine image" which is basically a complete copy of the OS, programs and data of the machine that you want to run in Amazon's cloud. Imagine you wanted to backup your computer's entire hard disk so that you could reconstruct the entire system - this is what you would need. You will replicate this image on one or more of Amazon's cloud machines.

I highly recommend following the AWS tutorial. It covers everything you'll need to know and doesn't have any distracting details. You will learn how to use images that Amazon have pre-made, how to set them up, how to change them, how to save them, and how to kill them.

Step 3 - Create your own AMI
For this, I started with the most current Ubuntu AMI at alestic.com. Some nice guy (Eric Hammond) has created a bunch of basic AMI's that have nothing more than a simple OS. From here, you will need to install all the programs that you're going to need. As someone who wasn't very familiar with Unix administration, this seemed daunting but it was surprisingly easy and I like Ubuntu a lot now. For example, this page shows you that, by typing 4 lines, you can get an Apache web server and PHP running (this is all I needed for Herd It). You will also need to copy all your data onto the AWS machine using FTP or scp (for example, I copied all the PHP and Flash files that make Herd It work).

Once you've got everything on the AMI running as you want it, you'll need to bundle your AMI and copy it into your S3 bucket. Again the AWS tutorial covers all of this.

Step 4 - Elastic DNS
This was the trickiest part... If you only want to run a single instance then you don't need to worry about this. But, the whole point of AWS is to let you create many instances to power your new web app that's going to take over the world. To achieve this, I found some help from this page but there was still a bit of work to do:

Step 4a - Register your domain.
There are a million places where you can get the domain "mykillerapp.com" or whatever.
I got www.herdit.org for $9/year from NameCheap.com
(I just discovered that, of course, there is already a site a "mykillerapp.com" and that it's a sweet applied maths quiz! For the rest of this tutorial, I'll just refer to my domain: herdit.org).

Step 4b - Set up a DNS forwarding service.
The domain name of all your new EC2 machines will be something like
http://ec2-a-bunch-of-numbers.compute.amazonaws.com/ and
http://ec2-more-different-numbers.compute.amazonaws.com/
In order for these to all map to your new domain name, herdit.org, you need to set up DNS forwarding. For this, you need a DNS service provider. You domain name service may provide this but, whether or not it does, ZoneEdit is a free service that gets the job done. You will need to transfer the DNS from your domain name provider and set up ZoneEdit (or whatever DNS service you use) to handle your new domain (it may take a day or two for these changes to register).

Step 4c - Tell you AMIs to register with your DNS service
Once your DNS service is running, you want it to forward requests for your domain ("herdit.org") to the EC2 machines ("ec2-0112358132134.amazon.com", etc.). To do this, you need those EC2 machines to tell the DNS service that they are ready.

The DNS registration works using a program called "ez-ipupdate" that you can install on your (Ubuntu) AMI by typing:

sudo apt-get install ez-ipupdate

Now all you need to do is to get ez-ipupdate to run whenever you start a new instance so that you don't have to log in manually. The Spatten Design blog has a good post on how to do this using Ruby. However, since I don't use Ruby, I wrote an init.d script that you will run when the instance starts. Copy the following and save it on your AMI as '/etc/init.d/update-dynamic-dns'

#!/bin/sh

### BEGIN INIT INFO
# Provides: update-dynamic-dns
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Default-Start: 3 4 5
# Default-Stop: S 0 1 6
# Short-Description: Update dynamic DNS on startup
# Description: Uses ez-ipupdate to send the current Dynamic IP address
# to ZoneEdit Dynamic DNS provider
### END INIT INFO

# Author: Luke Barrington <lukeinusa@gmail.com>

DYNAMIC_DNS_CONFIG_FILE=/etc/ez-ipupdate/dynamic_dns.yml
AMAZON_INSTANCE_DATA_ADDRESS=http://169.254.169.254
API=latest

# Read current instance URL from Amazon service
IP=`curl $AMAZON_INSTANCE_DATA_ADDRESS/$API/meta-data/public-ipv4/`
echo "Instance Dynamic IP Address = $IP"

SERVICE=`sed -n -e "s/^service:[ ]*/\l/p" $DYNAMIC_DNS_CONFIG_FILE`
USERNAME=`sed -n -e "s/^username:[ ]*/\l/p" $DYNAMIC_DNS_CONFIG_FILE`
PASSWORD=`sed -n -e "s/^password:[ ]*/\l/p" $DYNAMIC_DNS_CONFIG_FILE`
HOST=`sed -n -e "s/^host:[ ]*/\l/p" $DYNAMIC_DNS_CONFIG_FILE`

case "$1" in
start)
echo "Using dynamic DNS service = $SERVICE"
echo "Connecting with username = $USERNAME"
echo "Mapping IP to host = $HOST"

# ZoneEdit server name has changed since ez-ipupdate was last built
if [ "$SERVICE" = 'zoneedit' ]; then
eval "ez-ipupdate --address $IP --service-type $SERVICE --server=dynamic.zoneedit.com --user $USERNAME:$PASSWORD --host $HOST"
else
eval "ez-ipupdate --address $IP --service-type $SERVICE --user $USERNAME:$PASSWORD --host $HOST"
fi
;;
*)
echo "Usage: update-dynamic-dns start"
;;
esac


You will also need to create a file at '/etc/ez-ipupdate/dynamic_dns.yml' (or whatever you call it in the script above) that contains the following:


# service should be one of the services supported by ez-ipupdate.
# Possible values: null ezip pgpow dhs dyndns dyndns-static
# dyndns-custom ods tzo easydns easydns-partner
# gnudip justlinux dyns hn zoneedit heipv6tb
# (The above list is from man ez-ipupdate)
service: zoneedit
username: YOUR DNS SERVICE USERNAME
password: YOUR DNS SERVICE PASSWORD
host: YOUR HOST NAME (e.g., herdit.org)


Finally, you can run the update-dynamic-dns script by typing:

./etc/init.d/update-dynamic-dns

To register this init.d script to run automatically at startup, use this command:

update-rc.d update-dynamic-dns defaults

Now, as soon as the EC2 machine boots (well, after a few minutes), it should register itself with your DNS service and tell it to send requests for "herdit.org" to its address (e.g., ec2-123456789.amazon.com). The cool thing about ZoneEdit (or any DNS service that has "round robin" DNS) is that, if multiple machines all register to the same host, the DNS service will send requests to each one in turn. This will spread your millions of users across all the AMIs that you run.

At this stage, you will want to bundle up the AMI again. Now you are ready for Step 5...

Step 5 - Try and take over the world
Run hundreds of instances, pay thousands of dollars to Amazon, get millions of users, sell your site for billions of dollars.



Notes and next steps
Now that I''ve set all this up, we are testing Herd It to see how it can handle the load of many simultaneous users. Herd It users a Java server to coordinate everything (via XML events) and saves all the info in a MySQL database. These are both still running on my local server. I plan to put them on AWS sometime as well and I expect that this tutorial will help with that.

There are apps out there that can monitor your site's traffic to automatically create or kill new instances based on your traffic but, for the moment, I will be monitoring it manually.

Now, after all that work, why not go and play Herd It?!

Nöjeströtta älskar nya tal

News of my talk at the Math Club last month has been reported in Dagens Nyheter, "the New York Times of Sweden". Check out the article by Caroline Hainer - you can even see a picture of my arm pointing to some nice math!

In case your Swedish isn't as good as mine, here's the Google translation.
Note how I am described as a " long male, redheaded researcher" and Conor Deasy is now Conor PRESIDENT!


My next academic talk will be delivered in the style of the Swedish chef...

Math Club

I'll be giving a talk on "Machines that Understand Music" at the LA Math Club on Sunday, November 19, 2008. If you're in Hollywood, come along and meet all the musos and industry types that I hope will show up.

The Math Club is organized by the genial Roni Brunn, aka "the Girl From". Previous speakers have included Futurama's David X. Cohen and heaps of smart profs. I'm hoping to match the former for smarts and the later for humour...

Here's the abstract:

Humans can identify that a radio station is playing country music in less than one second (and switch channels!). Although the amorphous details of music and the emotions that it evokes in us are sometimes subjective, there are many concepts (e.g., instrumentation, genre, tempo, ...) that most listeners agree on. Given these statistical regularities, it should be possible to build a machine that can analyze and understand many aspects of music.

I will talk about and demonstrate a computer audition system that understands music. Using signal processing analysis of audio waveforms and machine learning models to identify patterns in the signal, my "musical search and discovery engine" goes beyond artist and song name search and can find "funky music with a horn section for a party" or "jazz saxophone for romancing". The system can also associate new music with relevant semantic tags, creating automatic record reviews. This musical search and discovery engine has many applications for personalized discovery and distribution of all music online.

Finally, I will explain how the data used to train the computer audition system to understand music is collected using an online music annotation game that is about to be launched on Facebook. Bring your laptops and we can all play together!