Posted by kyle on October 17, 2006 12:52 PM | bookmark / share: |
Last week I showed you a tutorial that I put together for The Ajax Experience, where we built a force directed model from the contents of an XML file. Today I'm presenting an example with full source from another tutorial in this talk, where we make a complete, interactive application that models music recommendations using Amazon.com's Similarity API. I'll be covering this example in detail in my talk next week at The AJAX Experience. You may view, download, and modify the source in this example. It's available under a Creative Commons Attribution License.
This application will enable a user to search for music by artist. When the user selects an album, they'll see a graph model reperesenting the album's relationship to similar albums, based on Amazon.com buying history. Here's a screenshot:
Click here to use this example application...
A Note on Aesthetics
Sure, album covers are square, but circles work better in 2-dimensional, dynamic graphs because they don't have any corners that might obstruct our view -- I think they look neat, too. Credit for this concept belongs to the authors of MusicMap, who created a similar application in Flash in 2004.
Example Source
In the tutorial I posted last week, I introduced the JSViz API and demonstrated how to construct a model by implementing a "Loader" to populate a DataGraph and a "NodeHandler" to control the representation of nodes and edges in our model. I've done the same thing here. Here's the source used in this example:
- Amazon.js: Loader and Search interface to Amazon.com E-commerce API.
- HTTP.js: A simple AJAX HTTP client.
- index.html: The HTML and JavaScript that tie the application together, including the NodeHandler.
Of course, this example also uses the JSViz 0.3.1 API. While I work on a permanent home for project files, you can access them in this post from last week.
Extending This Example
This example provides base functionality that you can use to build a rich, interactive application. You might extend or enhance the application by:
- Providing album details and linking to Amazon.com, ITunes, or other retailer product pages
- Linking to sample audio tracks
- Using node size to convey meaning (ie: Amazon.com SalesRank)
- Handling error conditions, like "no albums found"
- Create circle images using JavaScript (I'm currently using PERL/Image Magick)
- ...or atleast use an Alpha Image Loader filter to support transparent backgrounds in IE
Comments
Pretty impressive :)
I have read on Ajaxian with regard to your talk at Ajax Experience:
"The argument against data visualization in JavaScript is obvious: it’s intense stuff. Potentially millions of computations running continously....The answer to the question “Where do we have available CPU cycles?” may be “The client”, but unfortunately “The client” is also the answer to the question: “Who lacks the patience to wait for CPU cycles to complete?"
Well, how about grid-computing?
Just like SETI.
It may be just a crazy idea but could it be a way to share "resources" (in this case CPU-cycles) by individual users, a la P2P way?
Posted by: Kassad | October 23, 2006 2:04 PM
tried to make the tutorial work in safari. Looking at the error codes i found the problem at line 262 in ParticleModel.js
update: function tick()
I replaced this with update: function()... maybe you can check if introduces a new problem. But it works for me. Even in safari.
Posted by: Stefan | October 23, 2006 4:40 PM
I was trying your Jvix music application and it works well, but when I tried set it up on my server it didn't work. I noticed several calls to local .pl files in Amazon.js. Can you please explain what those files do or send me a copy of them. I am one of those guys that needs to take something apart to understand how it works.
Posted by: craig | October 23, 2006 4:51 PM
Stefan, Great find! Looks like I had a typo. I've patched the 0.3.1 code to reflect the change. Thanks!!
Posted by: Kyle Scholz | October 23, 2006 5:18 PM
Craig, Right. Since XMLHttpRequests can't be fired across domains, I needed to implement a local "proxy" in PERL that takes in user requests, relays them to Amazon, then returns the reponse to the user. I also use the proxy scripts to cache responses so I don't need to send a bunch of duplicate requests off to Amazon.
I have another script that renders the "circle" images using PERL/Image Magick (yes, that's the correct spelling). It's certainly not a requirement, but it resizes images a little better than the browser would on it's own.
I'll share the code for these when I get a moment.
Posted by: Kyle Scholz | October 23, 2006 5:25 PM
Kyle, thanks for the response. I have attempted the local proxy myself to no avail, I am eager to see how you did it.
I was working to create a version of Music Match when i came across yours via Ajaxian - my hat is off to you sir, as you created not only great libraries but an awesome javascript version of Music Match!! On top of that, you take the time to explain it and demo it to us old school C and perl programmers :)
Great job!
Posted by: craigl | October 23, 2006 6:32 PM
Craig, I merged the proxy scripts into one for convenience. You'll need to update your copy of Amazon.js. You can find the new "query.pl" here. This script requires LWP and MySQL libs from CPAN.
I included the caching code. To use it, change the values in the "connect" function to point to your MySQL server. Here's a copy of the table schema for "ProxyCache". If you prefer not to cache, just remove the lines that refer to MySQL, including the conditionals that determine if we should request the page. I hope this does the trick. I'll post the image rendering code when I have a free moment.
Posted by: Kyle Scholz | October 23, 2006 11:57 PM
Kyle,
Thanks for all the help. I am amazed at how simply you did the local proxying for ajax (xmlhttprequests). I definately was going about it differently. Like your js libraries - brilliant! I look forward to see how you did the image rendering. I use imagemagick on my home site via a perl script to give me the reflection look I am wanting so I am always curious how people use that tool. I feel it is one of the most underrated image tools out there (granted there is quiet a learning curve to using it well)
Posted by: craigl | October 24, 2006 9:06 PM
Is it possible to add/remove nodes dynamically to an existing graph?
Posted by: LFReD | October 28, 2006 11:50 PM
I have it kinda working, but I am definately doing something incorrectly. It could be how I stubbed out the image code just so I could mess with the amazon and xml portions...wicked good fun!
Posted by: craigl | October 29, 2006 10:19 AM
Craig, Sorry for my slow response, it's been a marathon of a week. For images, I'm using two scripts.
- image.pl performs proxy operations
- makeImage.pl fetches album art and constructs the images you see in the demo
The proxy relies on this SQL schema.
The image builder relies on two files:
- circ_mask50.png is a monochrome mask
- circ_border50.png is an overlay that adds a circular border.
Posted by: Kyle Scholz | October 30, 2006 12:32 AM
LFReD, It's easy to add nodes to the graph dynamically (see the implementation of NodeHandler). You can remove nodes, but the 0.3.1 API doesn't currently provide a simple method for doing it. You'll need to explicitly remove all forces that reference the particle from the springs/activeSprings and magnets/activeMagnets collections in the ParticleModel then remove the node from the particles list. I'll add this to the "todo" list for 0.3.2.
Posted by: Kyle Scholz | October 30, 2006 12:43 AM
Kyle,
thanks for the help! I hope this week is better for you.
I still am erroring in Amazon.js as xmlDoc is always null so i am still debugging that fun.
Is this a site specific key AWSAccessKeyId?
Thanks again!!
Posted by: craigl | October 30, 2006 2:53 PM
I got my AWS key and am trying to get xmlhttprequest to work ...
Posted by: craigl | October 30, 2006 3:43 PM
Error: 'Permission denied to set property XULElement.selectedIndex' when calling method: [nsIAutoCompletePopup::selectedIndex] = NS_ERROR_XPC_JS_THREW_STRING
HTTP.js
Line: 95
Any ideas on this?
Posted by: craigl | October 30, 2006 5:36 PM
Craig,
It sounds like you had a busy day with this :P. As far as I know, AWS Keys are not site specific, but it's certainly a good practice to use your own. Can you access your "query.pl" script directly? You should see valid XML, like:
http://kylescholz.com/cgi-bin/projects/amazon/query.pl?artistName=Bonobo
I'm not familiar with the Permission Denied Error you mention -- It sounds like a Mozilla internal component blew up ... Are you implementing this inside a Mozilla/Firefox plug-in? I'm not sure why you'd be seeing XUL and XPCOM errors as a result of anything you run from a web page.
Posted by: Kyle Scholz | October 30, 2006 9:56 PM
I almost got it working. I made some changes to makeImage and I removed the get_string from all functions in lieu of using CGI. I call makeImage instead of image.pl as well. I will gladly send you my changes if you are interested. I do not use image.pl at the time, though I plan on incorporating it into the makeImage portion.
I am having an issue with similarity though. If I enter an artist I will get the listing of all of their albums, but only the top option will generate a mapping of similar artists. The others in the list simply create a mapping of all the albums by the selected artist. Any ideas on this?
I have turned a couple of my compatriots on to your libraries...great work by you!
Posted by: craigl | November 1, 2006 2:46 PM
Kyle, can you explain me step by step: how I can create force directed graph (like http://www.kylescholz.com/blog/2006/10/modeling_music_recommendations_in_javascript_with_jsviz.html ) without AJAX.
Thanks
Posted by: Denis | November 8, 2006 2:03 AM
Denis, I can probably do that. It'll involve some CGI that grabs all of the data we need on page load. Is there a specific reason you're trying to avoid Ajax?
Posted by: Kyle Scholz | November 8, 2006 2:51 PM
Hi, my name is Malin. I´m studying the second year at Linkopings University in Sweden and working with a project where we´re going to have a webpage.
It seems like you know about how to create a vizualization graph. I´ve been trying to create this example "http://flare.prefuse.org/demo" (choose Layouts and then Force) folowing this tutorial "http://flare.prefuse.org/tutorial". But I got stuck.
Is there anyone who whant´s to help me creating a simple effect?
Of course your name would be vizible at the website.
Thanks
Posted by: Malin | April 22, 2009 3:00 PM
Hi, my name is Malin. I´m studying the second year at Linkopings University in Sweden and working with a project where we´re going to have a webpage.
It seems like you know about how to create a vizualization graph. I´ve been trying to create this example "http://flare.prefuse.org/demo" (choose Layouts and then Force) folowing this tutorial "http://flare.prefuse.org/tutorial". But I got stuck.
Is there anyone who whant´s to help me creating a simple effect?
Of course your name would be vizible at the website.
Thanks
you reach me at projektcyborg@gmail.com
Posted by: Malin | April 22, 2009 3:02 PM