Anyway, I saw a link for a video on YUI’s CSS component and I was just a little bit interested. I though it was probably just another CSS reset. There was some vague promise of Grids and I couldn’t imagine how they could make that easy so I still didn’t think it looked promising.

Wow was I wrong. If you haven’t seen this then you need to go check it out. Nate Koechley and crew have done a seminal piece of CSS work here. Here are the highlights:

• They provide a reset for all styles and fonts in all major browsers, including IE6. You are leveraging all the cross browser testing done by Yahoo
• Everything (fonts, boxes etc) is sized with em’s or percentages of em’s so the entire page will scale gracefully for 508C
• The Grids can be nested. You can do 2, 3 or 4 columns in a variety of width ratios (e.g. 2 columns: 1/4, 3/4 )
• Its really simple to use

At work we have struggled with CSS. We don’t have a full time web designer to work for us. So the devs do all the html & css. I’m fairly handy with this sort of thing but have no where near enough time to produce the level of product that you can get with YUI. Other devs have even less patience for this sort of thing. For us this make a whole lot of sense.

We are really sensitive to the semantic markup of the page and the use of em’s for 508C support. We want it to be nice to use not just live up to the letter of some spec. I started using em’s when we first did the style sheet but I was about to give up because it was getting so difficult to use. Now its going to be easy to provide the kind of experience and cross browser compatibility that I have wanted all along.

Just go to Yahoo’s home page and increase the font size. Look at how smoothly that works and realize how hard that is to set up. Now realize you can do that to your site and still get work done faster than before. Your developers can pick this up in an hour or two and start building better stuff.

Some parts of the iTunes API are shamefully slow. The Enumerator for ITrackCollection operates in n ² time. So the time it takes to walk the list grows quadratically with the length of the list. Iterating over the list with python and calling Item is actually faster that the enumerator. That was a bit of a shock. Item() is definitely n ², access at the end of the list is much slower than at the start of the list.

Even if Apple were to fix this we are talking about 5 seconds here which would give a max speedup of 2.2 seconds. Thats still too slow for search as you type. Besides, have you ever seen what happens when you drop a 5k rows worth of table into a web page? It won’t render correctly on anything.

The real solution is to page the results. This way access to the results is fast even in the worst case scenario at the end of a large list. This is what you would do in a real GUI application anyway. Never show the user more information that then can digest and Never do processing for anything you didn’t show the user.

The search for answers to Inrdio’s speed problems lead me down a path of optimizations that tightly coupled the iTunes stuff to the web server. Some of the optimizations were good but I’m not going to pay for them with bad architecture. So I took a step back and completely abstracted everything behind three classes: AudioPlayerInterface, TrackList, & Track.

The AudioPlayerInterface supports searching, queuing, and provides information about what songs will play/are playing. The TrackList supports the paged access to tracks, either from search results or from the playlist. Tracks are, of course, tracks with information like Artist, Title and so on.

In the iTunes implementation of TrackList is extremely fast. You construct a TrackList without having to iterate through the tracks. Then the web application module can get a page of tracks and compose them with the templating engine.

Access to Track data is done as its requested. The absolute minimum number of COM calls are made to render each template. The Track interface uses the wrapper pattern so its thin and fast. This way the tracks can support a large set of properties without having to copy each one into a temporary variable. The extra fields can be used by other templates, like the one that will render what playing now.

Expect a release (with code) as soon as I get the obvious bugs out.

innerHTML is a read only property in IE for the table and tbody elements. Trying to set it causes a very nondescript error with a code of 0. Firefox isn’t totally off the hook either. With large sets of rows there are rendering errors at the bottom of the table if you don’t define a background color. I’m using prototype.js to insert the rows returned from an AJAX call. I kluged a fix for IE by inserting the entire table every time.

Performance building that table was very bad for large result sets (1k+ rows). Python had a built in profiler & CherryPy has profiler support. I totally love this, this rocks 1000% more than Java. Reading and building the data set from iTunes was about 1/10th the total execution time so something else was not very fast. Turns out it was a module for the template engine that only worked with 2.4. I upgraded everything to 2.4 and performance got drastically better.

Now the major bottleneck is with fetching values from iTunes via COM. Thats about 80% of the time is spent in __getattr__ and _ApplyTypes_. Thats the COM stuff translating returned values into Python types and the majority of that time is probably spent in iTunes. The stop gap to give reasonable performance is to limit the number of rendered results.  I picked a ‘magic’ number, 300, which renders in under 2 seconds on my system. This will suck if your trying to search for music by genre and there are 1K+ results for ‘Trance’ in your library. Some sort of paging solution may have to be used to get through a result set like that. I don’t really see that as a major usage pattern in my experience. People generally ask for music by title or artist and this turns out to be very specific and returns a limited number of results.

• Small; as in not large, as in less than 1 MB compiled
• Embedded; must run from inside the app, no external servers please
• Must not stop me from talking to COM
• Must be nice to Unicode strings
• Must be nice to AJAX
• Must serve some static files
• Must server some dynamic content probably composed with a template system

Turns out there was only one system that I could locate that fit the bill, CherryPy. Getting the demo up and running was easy. Getting it to talk to COM was trickier. Its got something to do with threads and CoInitialize-ing them. The docs there almost have the magic formula, you have to hook on to whatever COM object you are calling after you call CoInitialize(). This means you can’t start up COM in the built in __init__ function of your root class in CherryPy, you have to call it from in the InitializeCOM()function, after the call to CoInitialize(). With that sorted I got the HTTP server to stop giving 501 errors and spit out the version of iTunes; “6.0.1.3 “. So far so good.

Now I am a big believer in the MVC pattern, code should stay in the code, and the view should be just the view. So next I wanted a template language to put together with CherryPy to complete my tool set. There seem to be a lot of these out there, mainly that want to mess with the HTML your writing to make sure your doing it right. Now, writing XHTML strict isn’t a problem and there is the posibility that I might generate some JavaScript code. The template engine has to be general, small size is a plus, easy to learn is a plus. Cheetah looks almost exactly like Velocity to me, and I’ve worked with Velocity before so its a good fit for a rapid prototype.

The bad news is that some of the cool stuff that iTunes is doing in the UI is not available through the COM interface. Of particular interest to me was the Party Shuffle playlist. You can add tracks to it but only at the end i.e. No “Play Next in Party Shuffle” via COM. You can’t determine how many previously played tracks are showing (unless your actually playing that playlist and work it out from the currently playing track). You can’t set the playlist that feeds into the party shuffle or any of the other parameters. These things are not blockers for what I wanted to do but they would have made things much nicer for users.

Then there is fun with ID’s. The docs say that all objects in iTunes support four script friendly functions to return the Source, Playlist, Track & Database ID’s respectively. If you have a problem accessing these functions its because the Source, Playlist and Track functions are declared to start with common letters; sourceID(), playlistID(), trackID(). The database ID function is capitalized as per the docs: TrackDatabaseID(). Also of interest, Python can handle the multiple return values of GetITObjectIDs() by turning them into an array. Score one for Python! Score another one for the Python COM browser which helped me track down the capitalization bug.

iTunes.GetITObjectByID() doesn’t work if all you have is the Database ID, which makes knowing that basically useless. To make GetITObjectID() work you need to know all 4 parameters. That isn’t exactly congruent with the docs. Supplying 0 to any of the parameters throws some sort of error.

The final twist is with Python. It supplies base types when a function returns a derived type over COM. So you have to cast the return type to the type you expected. This had me stumped for a while until I saw this Simon Brunning’s code. win32com.client.CastTo() is the magic bullet that fixes this. I wish there was some good docs for the Python COM interface.

Stay tunes for the next installment where I reveal even less about what Indrio might be or do or why you should care…

Download iTunes Folder Sync 0.2

Here is a link to how this script got started and how to use it. As always you need the Windows Scripting Host (aka wscript) installed and the .js extension set to be executed using \windows\system32\wscript.exe

These enhancements were made possible by Howard Darwin who did the beta testing and contributed ideas. He has a wish list for future enhancements that I would love to accommodate but I’m not sure that the iTunes SDK makes this practical with the WScript engine.

They had an off island firm (cyber-dynamics.com) code the site for them. It’s a proper news site with articles on their own pages etc. They still don’t pepper articles with the classifieds but perhaps that will come in the future.

I wont bother to regale you with a description of how livid it makes me as a technology professional to see my country misrepresented in this way. I will defer to Mr. Aaron Boodman, who already keeps a hit list for people who deserve execution for their negligence. His article details the problem and its very simple solution.

I have also found a way to make www.gov.ky your own personal bitch by rewriting their broken java-script on the fly using the fantastic greasemonkey plugin for Firefox (You do have Firefox, don’t you?).

My first move was to add a configuration option to have the client act as an rdv-server. This stops the client from running any services. It loads the groups and becomes an RDV in each. Then I had some more ideas about automation:

• It would be nice to launch one or more peers (say up to 4) from within Eclipse. I could do this either with Eclipse or with Ant. It would also be nice if the files for these clients didn’t clutter up the main development directory.
• It would be nice if each client would remember its position on screen. (sometimes its the simple things)
• I still don’t have a good idea of how to write JUnit tests that use the client when its online. JUnit seems naturally useful for simple things but for network interaction testing I’m not sure how it should be used. I think at least the client would have to be headless (no GUI).
• Restart JXTA from within the application, optionally erasing the config data. There may be issues with this, i see developers having trouble doing this on the dev list.
• Have the client display what version it is, and write that to the log file at startup. I’m not sure where that should be stored. The version numbers are generated by Ant at build time. It would be nice if I could retrieve it as a property of the mindshare.jar file.
• Peer initialization times aren’t consistent, mainly in joining the NetPG. If only there was something I could do in code to ‘encourage’ each peer to find an RDV. Starting my own NetPG is an option, perhaps just on my LAN using multicast. I don’t like testing things on the LAN though, multicast hides problems that don’t crop up till you use the program across the internet.

Other stuff unrelated to testing:

• RDV timeouts are 2 minutes! That’s way too long for a real world application. The JXTA guys need to remember that people will hardly sit still for 7 seconds while a web page loads. It would be nice if I could get more fined grained control of the RDV service and perhaps speed things up. Empirical evidence seems to indicate that 15 seconds might be as fast as the current implementation can manage.
• It would be nice to have a bug tracking database to testers could report bugs. Either as a forum of a dedicated bug tool. (Bugzilla perhaps?)
• It would be nice to have source control. I have the history in Eclipse at the moment which is all I have needed so far but source control would be better. Subversion has been suggested as an alternative to CVS, but Eclipse has nice CVS integration.
• I need to update this website. Its not nice to look at and it needs a better description of the project.

All peers belong to the “World Peer Group”. I presume this means that all peers can communicate with each other because they belong to this group. Furthermore all peers are encouraged to rout traffic for each other. As a peer I would not be inclined to rout traffic unless it directly benefited me or other peers in my group. The last thing I want to be doing is routing traffic for some file sharing service when I’m trying to commit to changes to the group. My traffic comes first, preferable exclusively. The only thing I would agree to route for other peers would be discovery messages. Using the network at large to help people find each other is a fair exchange in value for traffic.

I’m aiming for a secure network. It seams peers can join my group without authenticating with one of the group members first. I want to have a custom authentication protocol with public key encryption.

There dost seam to be clear answers to these questions on the JXTA website either. I guess its time to dig into the protocol spec to find out.