Saturday, November 14, 2009

mmm, analog pie

My office, the official HQ of the UCSD Computer Audition Lab, is full of servers, monitors and academic papers. But it was decidedly lacking in cool musical and electrical devices necessary for a real "laboratory". All that changed last month when I finally fixed my Paia Theremax theremin and Brian setup his old Paia 4700 analog synth.



We've been making some pretty crazy sounds (I hooked the theremin output to the synth's filter for some awesome effects) and finally, we've got a lab worthy of the mad science we do! The guys in the office next door aren't quite so excited...

Most bands are used to getting their reference tuning from the piano / keyboard player. But one of the trickiest tasks was tuning the keyboard:

Adjusting a potentiometer behind each key changes it's output voltage and thus its pitch. Of course, then you still have to tune the oscillators... I don't know how anyone ever performed with these things.

Wednesday, November 4, 2009

Google Music... what's the big song and dance?

I posted recently that Google's Music search has launched. Unlike most Google products (search, GMail, Maps, and, eh, Wave?), it's kind of underwhelming. You type in the name of a song or an artist and you get to listen to the song.
Once.
Maybe - sometimes you only get a 30 sec clip.

I can get 30 second clips (of more songs) on iTunes.
I can get full songs on LaLa, GrooveShark, Rhapsody/iLike.
I can listen to as often as I want on YouTube, TubeRadio, blip.fm and Spotify (if I live in Europe or am a hacker).
I can even download them (in case you didn't know) on BeeMP3, MP3Raid, BitTorrent, LimeWire, Gnutella, eMule, etc. etc. etc.

And I'm not the first to complain about this watered down, industry serving jive.

So what's the big deal?

I would say that Google's music offering in the US is music search - if you know what you're looking for, Google will help you listen to or (they hope) buy it in a nice, familiar gInterface.
What I'd like to see - and what Google have launched in China to compete with search leader Baidu - is music discovery. I want to be able to search for "funky bluegrass that will get my hoedown party going" or "mellow romantic tunes that will get this girl to fall in love / go to bed with me". Google can't do that.

-- "Where can one search for music with words", I hear you ask?
-- Why, Herd It's music discovery engine of course!
I've built a music search engine where you can find any music, even unknown, MySpace, long-tail, no-hopes, by using words to describe what you want to hear. It's going to change the way people discover music. The discovery engine is in Beta testing right now so sign up to check it out. Or, to get a taste of what we're up to, you can try the fabulous Herd It color search demo - it's magically musicalicous!

Wednesday, October 21, 2009

Google for Music... by Google?

The big goal of my research for the past few years has been to build a system that we liken to "Google for Music". Using signal processing and machine learning to analyze music, we can automatically associate any songs with hundreds of descriptive tags. This means that you can find music without knowing the name of the band / song / album you want - just by typing in words that describe what you want to hear. To check out a prototype of our system, go to HerdIt.org and sign up to be a Beta tester. While you're there, play Herd It and make the search engine even smarter!

Now it seems that Google is finally getting on board with this themselves. After years of staying away from music (the legal, licensing and logistical difficulties scared even them), it seems like Google music may be coming soon.

Now, all we need to do is get them to use our algorithms to make their search even better!

Modelling Music as a Dynamic Texture

My paper "Modeling Music as a Dynamic Texture" has been accepted for publication in IEEE Transactions on Audio, Speech and Language Processing! This is work with my colleague and good buddy, Prof. Antoni Chan, and my advisor and good buddy, Prof. Gert Lanckriet and is awesome for at least 5 reasons:
  1. It presents a new, statistical model of musical audio that really takes time into account
  2. It introduces - and solves - the "Bohemian Rhapsody problem" (in case you don't know what this is - try describing the sound of BoRhap in 5 words or less...)
  3. Our model can automatically separate the verse, chorus, solo, bridge, etc parts of any song as well or better than anything else
  4. It will be a chapter in my thesis
  5. I get to talk about Freddie Mercury's falsetto, Bjork and the Beatles in an academic paper
Lookout for the paper in a library near you before the end of the year! We're working on extending the model to more Music Information Retrieval tasks like tagging and similarity. In the meantime, check out some of our results on automatically segmenting music.

Listen to the 35-second version of Bohemian Rhapsody!

Monday, October 12, 2009

Calling library objects from a loaded SWF

Herd It is a Flash application that loads lots of external SWFs - to play minigames, show results, etc. - into the main SWF loaded by the player. The problem is that, any object that is defined in the loaded, external SWF is not known to the main SWF in advance and so can not be referenced by the main SWF's ActionScript code.

When you load the external SWF as a Loader, you can refer to objects on it's stage using:
var myLoader:Loader = new Loader();
var myUrlReq:URLRequest = new URLRequest("data.swf");
myLoader.load(myUrlReq);
var myObject:* = myLoader.content["myTimelineObject"];

I always had thought that you couldn't access anything from the external SWF's library - until now! Thanks to this blog post by Koen De Weggheleire (another good thing to come from Belgium), I have learned that it IS possible to access the external SWF's library.
First, we load the loader:
var myLoader:Loader = new Loader();
var myUrlReq:URLRequest = new URLRequest("data.swf");
myLoader.load(myUrlReq);

Once the loader has loaded (you may want an Event.COMPLETE handler to wait for this), we can now find objects in the external SWF's library using the contentLoaderInfo's applicationDomain:
var LibraryClass:Class = loader.contentLoaderInfo.applicationDomain.getDefinition("LibraryClassName") as Class;
var myLibraryObject:MovieClip = new LibraryClass as MovieClip;

Nice one, Belgium!

Thursday, May 14, 2009

From a Queen Song to a Better Music Search Engine

My recent ICASSP paper described a Dynamic Texture Model that can capture both the timbral AND temporal qualities of music. An article about this and how it contributes to our music search project by solving the "Bohemian Rhapsody Problem" just got posted on the Jacob's School of Engineering Site, UCSD's site and EurekAlert.org!

Read the article here and check out how the system can automatically divide Bohemian Rhapsody into homogenous segments and then we can tag each segment individually, rather than treating the whole song as a single unit: