gSkinner - Home

Music Visualizer in HTML5 / JS with Source Code

Posted on March 3, 2011 by Grant Skinner

It’s no secret that I like building music visualizers, or that I’ve been playing with HTML5 a fair amount lately. Given that, I thought I’d combine the two interests, and build a music visualizer using JS, the canvas & audio elements in HTML5, and the EaselJS framework.

The primary challenge was that Javascript doesn’t have any built in mechanism for accessing the volume of a playing audio tag. To address this I wrote a little AIR application that will read an MP3 file using Sound.extract() and export peak volume data as a text or JPG image file. I then wrote a JS class called VolumeData.js that reads in these files and provides access to the data via a simple interface (ex. myVolumeData.getVolume(time) ).

With those pieces in place and tested, I started putting together a demo of it in action using EaselJS. Two of the newest features were compositeOperation support (which let me approximate an “add” blend mode), and the drawPolyStar method, both of which I used to excess.

I think the end result is pretty cool, though it requires a fairly modern system, and will still melt your CPU – I was intentionally pushing things hard to try to find the performance limit. It requires an up to date browser to run.

I built two variations: Star Field and Atomic. Occasionally dynamic audio loading seems to break on certain browsers, just reload if it gets stuck on “loading music”.

If you’re interested in building your own music visualizers in HTML5/ Javascript, you can download the demo source, VolumeData.js class (MIT licensed) and VolumeData AIR application here. Also, be sure to check out the latest version of EaselJS (we just released v0.3.2 today).

Please let me know if you build anything cool with the code. I’d love to see it.

Follow @gskinner on Twitter for more news and views on interactive media.
52 Comments

Great stuff Grant. I liked the pixel matrix (2 pixels for blue makes perfect sense as it's always darken on its own). Need to try EaselJS soon.

Posted by: Og2t on Mar 3, 2011 10:43am URL: http://play.blog2t.net

nice but 7fps.

Posted by: viaria on Mar 3, 2011 11:20am

What is that song? I know I've heard it somewhere before.

Posted by: Blacktiger on Mar 3, 2011 11:51am

you should try using flash, the frame rate will be much better and it will work on any browser

Posted by: hoitybeaver on Mar 3, 2011 12:55pm

Awesome Grant! I'm a huge fan of music visualizers as well. I prefer the Atomic version, it's a little more interesting I think, and more props on the song choice, perfect song for it. So does the peak volume directly relate to the effect, or is it the slope of the change in peak? It seems to react to change more than just volume, well, at the louder parts it stays pretty large too though. great fun!

Posted by: Evan on Mar 3, 2011 2:13pm URL: http://circlecube.com

Evan - it uses a combination of momentary volume, delta volume (over 0.2s), and average volume (also over 0.2s). Momentary volume provides an obvious tie into the music, delta let's you react to change, and average gives you a nicely smoothed value to work with for effects that you don't want twitching too much.

Posted by: Grant Skinner on Mar 3, 2011 2:23pm

thanks Grant, that makes great sense, you're too smart.

btw - I remembered where I'd heard that track: Inception

Posted by: Evan on Mar 3, 2011 2:27pm URL: http://circlecube.com

@hoitybeaver that would defeat the entire process

Posted by: dekz on Mar 3, 2011 9:58pm URL: http://dekz.net

Nice to see this happening!

Posted by: random on Mar 3, 2011 10:45pm URL: http://metacosm.com

Man! you are good with Visualizations.

Posted by: Tahir Ahmed on Mar 3, 2011 11:57pm URL: http://tahirahmed.wordpress.com

You're great as always!
It would be interesting to compare performances with the same application done in Actionscript

Posted by: Matita on Mar 4, 2011 2:48am

I like the unintentional inverse correlation between epicness and framerate. Keeping an eye on the fps meter was like

"twentysixtwentyfivethirtytwentysix NIIIIINE FRAMES PER SEEEEEECOND twentytwotwentysix..."

I kid of course. It's very impressive work, and looks great. As a VJ, I can't wait till we can use this on arbitrary incoming audiostreams in realtime...

Posted by: mediapathic on Mar 4, 2011 4:39am URL: http://mediapathic.net

Nice work!!
Have you seen this?
http://lab.simurai.com/ui/zen-player/

It works even on my iphone 3gs with 4.2.1 ios

Posted by: Pablo Cabana on Mar 4, 2011 10:32am URL: http://www.cabanacriacao.com/

Nice! But in my Opera 11.01 I can hear the music but instead of the visualization I can only see a black box. The error console logs this error every tick:

Uncaught exception: TypeError: Cannot convert 'volumeData' to object

Error thrown at line 158, column 1 in tick() in http://easeljs.com/demos/MusicVisualizer/index.html:
var vol = volumeData.getVolume(t);
called from line 31, column 7 in () in http://easeljs.com/demos/MusicVisualizer/easel.js:
j==null||b&&i||j.tick==null||j.tick(a)

Posted by: Wolgor on Mar 6, 2011 2:01pm

Grant - you really have jumped ship at the wrong time - here I am playing with molehill - new sound effects, new bezier system and huge, etc, etc in flash 11 - and here you are reverting back to what is essentially one step up from animated ascii scripts of the 90's...?

Just sayin......each to their own though.

p.s.

Really didn't work in firefox, ok in chrome (20fps).

Posted by: Aristophrenia on Mar 9, 2011 5:25am

Aristophrenia - I don't limit myself to looking at technology platforms as ships to jump onto or off of, I look at them as tools, where I can be happily building things and playing with many of them at the same time. Different tools have different strengths and serve different markets, limiting yourself to one is myopic. Have fun, and learn as much as you can.

Posted by: Grant Skinner on Mar 9, 2011 5:33am

True, this is also the right time to learn and get used to the new technologies while they are in their infancy. Funny though to see how different browsers deal with the rendering. If you run one of the visualizations in two browsers simultaneously, you will be using all your cpu. In Firefox 4 beta, memory just goes through the roof.. no hardware acceleration I guess :)

Posted by: Hassan Elousami on Mar 9, 2011 6:12pm

[...] визуализатор музыки на HTML5 / JS с исходным кодом [link]; [...]

Posted by: Что почитать … on Mar 11, 2011 8:57am URL: http://vyunev.wordpress.com/2011/03/11/%d1%87…

I see this library being picked-up by adobe, plugins are to hard to maintain why not let the browser vender's do the handwork and you build great IDE's imo.

Posted by: chris on Mar 21, 2011 1:44am

There are experimental open APIs for accessing raw audio data, including Mozilla's Audio Data API and a (chromium group?) W3C Web Audio API. The W3C Audio Incubator Group page has many more links to experimental HTML5 audio data examples.

Posted by: Colin Sullivan on Mar 21, 2011 1:57pm URL: http://colin-sullivan.com

[...] http://gskinner.com/blog/archives/2011/03/music-visualizer-in-html5-js-with-source-code.htmlShare this:   [...]

Posted by: Music Visualizer in HTML5… on Mar 25, 2011 2:05am URL: http://www.ilumindy.com/post/music-visualizer…

Nice visualizer! imo one of the most impressive examples with html5 js at the moment...

Posted by: mr wayann on Mar 26, 2011 12:27pm URL: http://srwayann.com

Grant - good to see you're still as busy as ever. I saw you speak at Flash Forward NYC. I also am a fan of visualizers and was floored when I tried someone's iPad for the first time and saw - "No Visualizer in iTunes???"
Cool stuff, I look forward to catching up on your site.
~Mike

Posted by: Mike Stuart on Mar 28, 2011 11:53am URL: http://www.stustustudio.com

> you should try using flash, the frame rate will be much better
> and it will work on any browser

This one comment explains everything that is wrong with the FlashPlayer Plug-in. It is *not* universal, it does *not* work in any browser, but some people just will not accept that. The only thing that is universal is HTML. FlashPlayer is only included with Windows PC's, and the latest trend in FlashPlayer is disabling it to prolong battery life or enhance security. Click-to-Flash won a design award last year. The IE6-IE8 users you are catering to with FlashPlayer are also the ones whose companies have disabled FlashPlayer for security reasons.

For the record, I viewed this music visualizer in Safari 5 on a MacBook Air that does not have FlashPlayer. Worked beautifully. Also, for the record, I've been a Flash developer since 1997. Flash peaked in 2005 and died in 2007. Since then, Adobe has been promising that it will rise from the dead "next year." Get over it!

Posted by: Hamranhansenhansen on Apr 3, 2011 3:18am

[...] http://gskinner.com/blog/archives/2011/03/music-visualizer-in-html5-js-with-source-code.html [...]

Posted by: April Meetup Details Anno… on Apr 4, 2011 6:40pm URL: http://austinjavascript.com/april-meetup-deta…

Fantastic! Looks really nice. Looking forward to seeing more Flash creativity bleed into the new (old) world of HTML! :-)

Posted by: moses gunesch on Apr 6, 2011 2:07pm URL: http://blog.mosessupposes.com

[...]  At first I thought I invented this method, but I Googled around and discovered that someone else beat me to it.  Nevertheless, it works well in practice and provides interesting [...]

Posted by: An HTML5 Music Visualizer… on Apr 30, 2011 5:18pm URL: http://andrewtwest.com/2011/04/30/an-html5-mu…

Hi Grant Skinner

This is time for rant from very loyal follower of you from many years.
You have been great inspiration for many flash community members. You've done great contribution to Flash community without doubt. Now I blame you. You jumping in HTML5 world supporting Apple void words and showing to many of your followers new direction. If Flash will die someday you are going to be responsible for this even more than SJ. Instead of putting your effort in rewriting Flash greatest invention DisplayList and porting it to HTML5 canvas (patent still partially belongs to Apple) you should make exactly the same thing in order to port Molehill 3D for 2D content. This is what we expecting from you. We all know, that Adobe implementation is not perfect in many places but a lot of Flash evangelist and enthusiasts make all this good things happened in the past. Pushed Adobe to improve things. Your steps now doing more damage into flash community that you can even possibly imagine or you are not aware of. I have removed your blog from my blogroll list and hope more flash developers especially flash evangelist will do the same thing. You are clearly working against Flash Platform. I don't know is being famous is your priority over being loyal to platform thanks to you became famous is more important to you.

You already know how weak JS and Canvas implementation is compare to AS3, so it is not about being performance lover. My heat is bleeding.

Posted by: devu on May 1, 2011 3:46pm URL: http://www.flaemo.com/blog

@devu +1, but

I have an idea:

MXML-AS3(Flex) should be have the native support by any browser as like as HTML-JS. (Posted on LinkedIn)

my comments:
In this case, it (Flex) will get a new impetus + HTML dev look to the side Flash + Flash to stay up(more) as the trend.

I know we has to close FlashPlayer today, but many of developers want to be have it, may be as OpenSource.

My question: What do you think about?

P.S. I collect the opinions of experts.
Being not engaged by Apple, Adobe, MS, Google, etc, I am expressing my own interests.

Posted by: chrm on May 24, 2011 4:47am URL: http://www.reflexer.org

The AIR app in the sources zip will not install on XP. Error message suggests contacting the author. Possible to repackage or provide source to rebuild?

Posted by: Ted on Jun 9, 2011 1:08am

[...] своем блоге в посте Music Visualizer in HTML5 / JS with Source Code Грант делится своим опытом в визуализации аудио с [...]

Posted by: Визуализация … on Aug 7, 2011 9:30am URL: http://blog.javadav.com/?p=1359

Hey,

Cool demo. I'm working for Opera. And right the visualizer doesn't work in Opera 11.50 and Opera 12 at least. There is an easy fix to make it work.

In http://easeljs.com/demos/MusicVisualizer/VolumeData.js


var instanceofImage = false;
try { instanceofImage = data instanceof HTMLImageElement; }
catch(e) { instanceofImage = data instanceof Image; }
...
} else if (instanceofImage) {

Posted by: karl on Aug 22, 2011 10:53am URL: http://my.opera.com/karlcow

I loved this, and it was great to see that my frame rate never went below 20fps the entire time, so great job.

Posted by: crackruckles on Sep 10, 2011 5:42am

@devu
what the heck is wrong with you? lol

Posted by: Joe on Oct 30, 2011 7:24am

[...] http://gskinner.com/blog/archives/2011/03/music-visualizer-in-html5-js-with-source-code.html [...]

Posted by: April Meetup Details Anno… on Nov 12, 2011 7:53pm URL: http://suyok.com/uncategorized/april-meetup-d…

Hey, What could be the solution for creating a music playlist with visualizations? and what can be solution for live stream music?

Posted by: Rohit on Nov 19, 2011 1:58am

No matter what you do, you never seize to amaze me! Keep it going.

Posted by: Christian on Dec 18, 2011 4:51pm

Hello Grant, I hope everything is fine with you. I don't know how I stubbled on to your page but, I didnt! I love "star Field and Atomic" so much, love you add your html code to my site. I am new to web building myself. making a code for myself that would be as dynamic as your is isn't possible at this time that I really need to have something like this. Let me tell you a little something about my site, http://exstreamjamsradio I am creating is a new kind of social site and online station. This site is where people like yourself can come and enjoy themselves with music being played in the background. This site I was originally made with (php) however, to be honest, me and php don't get along.(lol) So now I am creating a new site with something I know a little more about (HTML)(javascript). Currently, I am building this new site (which promises to be mind blowing), One that I think would really bring knew people to my station.
So, instead of uploading music to it it will monitor the ip and port that I would you could set in the coding.

Back to your Atomic visualization. The first question is could it be resized so that I can use it and place it wherever, I choose too? Whith the fact that myself, Is a dj on this site, Could it be used for the visualization to be streamed to it? Last but not least, would I have permission from you to use the exacted coding on my site? If I would have permission. I would advertise you as one of my links on your site with specail thanks?

One more statment and question, the new site of which I am creating has a brick background to it; if I was to send you the photo that I'm using as the background, could you implament that photo and the same background?

Thank you for your time in this matter and hope I will hear back from you soon. Please email me at my ChrisTurley@usa.com not twitter.

Posted by: Chris Turley on Jan 26, 2012 5:11am URL: http://exstreamjamsradio.com

[...] [...]

Posted by: 3 Breakthrough Ways to Vi… on Feb 6, 2012 5:04pm URL: http://www.sitepoint.com/3-breakthrough-ways-…

[...] his blog post Music Visualizer in HTML5 / JS with Source Code, Grant shares his experience with audio visualization using [...]

Posted by: 3 Breakthrough Ways to Vi… on Feb 7, 2012 7:20pm URL: http://resellerwebhost.net/3-breakthrough-way…

Gorgeous! Thank you.

Posted by: Dan Owen on Feb 9, 2012 12:24pm URL: http://ensightful.com

Awesome job! Running on an i3 2100 with intel integrated graphics I'm getting no glitches at all. Looks like you'll have to push the boundaries again :-D

Posted by: David Lawrence on Mar 15, 2012 7:11pm URL: http://dlawrence.net

for some reason, when I try this demo, the visual turn grey and everything disappears, but the song keeps playing. what's my problem?

Posted by: AlexGermanHTML on Mar 20, 2012 3:04pm

This is totally cool but we need more and be able to play our own music

Posted by: Finiderire on Apr 14, 2012 6:00pm

Clicking the demo links just takes me to the createjs site.

Posted by: Confidant on May 18, 2012 11:14am

I'd like to try this out, but the demo links redirect from http://easeljs.com/demos/MusicVisualizer/index.html to http://www.createjs.com/#!/EaselJS, and the demo is nowhere to be found. :(

Posted by: JacobSyndeo on Jul 6, 2012 6:27pm URL: http://bronynet.com/

I absolutely love the idea behind this, but provided that the sole purpose of the AIR app is to extract (semi)-precise data, saving in a lossy format like JPEG seems to be a bad idea. The nature of the JPEG format would seem to degrade the volume/pixel data. Why not use PNG or at least allow for it as an option?

Posted by: Kim Jensen on Jul 23, 2012 11:42am URL: http://www.litewerx.dk

I'm unable to add audio files through volumedata.air for some reason, can give steps how to do it? So that I can correct myself if I'm doing a mistake..

And is it possible to make this visualizer work on a radio stream?

Thanks in advance :)

Posted by: sam on Apr 28, 2013 1:05pm

[...] his blog post Music Visualizer in HTML5 / JS with Source Code, Grant shares his experience with audio visualization using [...]

Posted by: 3 Breakthrough Ways to Vi… on Aug 13, 2013 5:57pm URL: http://blog.production.sitepoint.com/3-breakt…

[...] Preprocessing the sound data into another format This one seems pretty crazy, but in a cool way. Grant Skinner made an AIR app that takes the sound data from a file and puts it into an image, which can then be [...]

Posted by: HTML5 Newgrounds Player -… on Oct 6, 2013 9:15am URL: http://www.pahgawks.com/webapps/html5-newgrou…

hi this is RAD

Posted by: d on Dec 23, 2013 3:03am

I download your source code, but when i use another music file, it not working! Why and how to use with another music file!

Posted by: dvd_nasa on Sep 18, 2014 2:40am