Next up is the links page which I’m hoping to replace with the contents of my bookmarks from FireFox. I also rather like the idea of doing some art work for a site logo and icon, perhaps its time to try The Gimp again.

Right now I need to do several things before development can continue. On the organisational side I want to change 3 things:

• Get the source code hosted in a Subversion (SVN) repository. SVN supports renames correctly for directories which is critical when you go to rename packages in Java, something I have no fear or compulsions about doing. We may eventually host downloads from Sourceforge but I have no interest in using CVS as the repository.
• Get a MediaWiki up and running to host all development discussion, like how they do for Firefox.
• Set up bug tracking with Bugzilla.
• Choose a Licence for the code and add it to all source files.

For an open source project those are the best in class tools. The SVN may be problematic because it requires special installation & configuration. The MediaWiki and Bugzilla should run from my current web host.

The other stuff is more about what direction the project should take. Particularly with regards to networking technology and JXTA. I got a profiler out and I am not too happy with what I see. Even small JXTA apps like MyJxta have a 50MB footprint (comparable to Limewire or Azureus) and they randomly chew up the CPU even when idle. JXTA also has issues with the speed of its DHT searches, aka the Discovery Service, which made it useless for presence support. It also takes a long time for each node to startup  and join groups. Interestingly JXTA is not I/O bound as one might expect, being a networking technology and all, actually its XML processing boutd, spending about 67% of its time munging XML documents. You get lots of cool stuff like NAT traversal ‘for free’ with JXTA, the question for me it just how free is it for my particular application?

I switched to Log4J from Java’s built in logging API. I am very happy with the switch but the log window in the app has broken. What I really want is a coloured log output in the log window. Something like this:

 This would make debugging at any location just a bit easier. The output would be in HTML of course and you could save the logs. Trouble is there appears to be no off the shelf solution for hooking up a log appender to a JTextPane. i wrote the Layout in about 10 minutes though. I may take another crack at this tonight because I really like the idea of being able to read the logs anywhere.

New:

• Work on BEEP connections between peers.
• Peers decide who is the ‘client’ and the ‘server’ by comparing the URI’s of their respective JXTA PeerID’s. The lesser PeerID is the client and thus responsible for making the connection.

Refactoring:

• All the old logging code is now replaced with Log4J.

Bugs:

• Fixed a subtle bug in Presence that happened when peers went offline and came back rapidly. The per record would be replaced and a timer task would remove a different but ‘equal()’ record.
• Fixed a synchronization bug in Presence shutdown where concurrent modification could occur in the peer list.
• Found a subtle bug in the buddy list display. If the list is interacted before all peers appear online it will not display new peers. Most curious.
• Also the buddy list is not being saved as it was before. This is actually a cool feature at this phase of testing so it may be made optional with the default being true.

Overall I like the Wiki over the mailing list or a forum so installing one is on my todo list probably in late February.

The other requirements are more subtle. Mindshare save everyones files to disk and cares about who owns those files, something no other P2P program cares about. The user ID may be called upon to disambiguate files in cases where two files have the same name or to store each users files in a separate directory. This puts severe restrictions on the characters that can be used as ID’s. For starters using Base64 encoding cant be used for file/directory names because it includes the ‘/’ character which is either illegal or delineates directories on Unix platforms and in URI’s.

Then there are the case-preserving, case-insensitive filesystems of the two popular desktop OS’s. Base64 is case sensitive so raw Base64 cant be reliable used because two different Base64 strings might be the same thing as a file name.

Finally Base64 uses the ‘+’ and ‘=’characters. The = is easy to avoid by packing the base 64 such that no filler is needed. The ‘+’ character falls into the punctuation group and is thusly not suitable for use in the authority section of a URI.

So for now Mindshare ID’s will be 15 bytes of secure random data, encoded with Base64, resulting in a 20 byte ID. All capitol letters will be replaced with their common equivalent (id.toLowerCase()). The characters ‘+’ and ‘/’ will be replaced with the more URI friendly ‘-’ and ‘_’ respectively. There is still the possibility of collision in this scheme by virtue of the loss of capitalisation. I consider this to be unlikely or at least as nebulous as the possibility of generating identical random bits because Java doesn’t have access to accurate spatial or chronological information. At some future time an encoder can be built that encodes raw bytes in the alphabet [a-z][0-9][-_] without changes to the protocol.

Basically this is a post to say I have thought about this issue but I am too busy/lazy to do the right thing at this point in time. This gets put off for 0.2 when we do crypto & security.

Give the time constraints the main priority is to get tree and file swapping implemented. Events have, however, conspired to make this not so easy. I lost some important code (formatted it into oblivion) that was not too trivial to write. To save time I am cutting XML as the back end data format. Right now much of the core already works on XML. The alternate encoding will be BEncoding much less stressful to work with. It solves many problems associated with sending binary hashes inside XML that cant contain binary characters. Right now I want to strip out all the XML code and replace it with BEncoded messages instead. The work needed to do this may be too much to be worth while for the alpha.

Presence support was rewritten to be more general. Integrating this back into the Mindshare client is a bit of a pain but the result should be a more responsive presence service.

All point to point links will use BEEP for transport. Before the code loss I had BEEP working over a JxtaSocket quite well. This uncovered a bug with closing JxtaSockets and BidiPipes that is now fixed. JxtaSocket still does not support keepalives so the protocol will include a keepalive packet. In the future a second BEEP channel could be used for one to one chat.

Heading up to the release I will try to post more often so check back.

Recently events have also conspired to push back my thesis defense until mid February. This is more time than I felt I needed to have a demo of the prototype ready. So given the slightly relaxed time table and my current set back I thought now would be a good time to take a step back and really consider what’s important on this project. I’m being too much of a perfectionist about things. I need to prioritize just a few things that need to be done the right way and let the rest slide.

Grouper is as close as anyone has gotten to the idea behind Mindshare. I’m sure that I could convince lots of my friends to use that, especially for swapping photos. I am also encouraged that it took a whole team of ex AIM developers to put that together. As the sole developer its hard feel that you are making any progress.

There are good things to be said for non collaborative sharing, like better control of your bandwidth. In the end Mindshare will support both kinds of sharing, thats actually what I wanted from the start. Mindshare will still enable you to do things you cant do with Grouper. Real collaboration takes cohesion and Mindshare gives groups a way to express themselves cohesively. Without a working prototype its hard to tell how users will respond to that idea so I will be following Grouper to see how well it is accepted.

Distributed BitTorrent is exactly what Mindshare will need for a back-end file transfer engine. Every peer will be its own ‘tracker’. For now though the tradeoff is speed of implementation vs transfer speed. BitTorrent is a proven technology but implementing it in Mindshare is a big risk in the next few months. It is obviously the right thing to do for performance reasons but it might break my time budget. The prototype will have a very simple file transfer engine that can be improved later. Better to have something quick and than something I am forced to abandon at the last second.

JxtaNetMap absolutely rocked for debugging. I could see my peers, their location on the network and actually observer network instability as it happened. The only thing that sucked was having to build it from CVS to make it work. Yet another powerfull ingredient in the JXTA ‘special sauce’, good stuff.

The next step is to test out the new JXTA BEEP session creator I’ve built. The idea being that a JxtaSocket should behave like any regular Socket once a connection is established. So for minimal work BEEP should run over JXTA :-). All in all a good days work.

Photos of the hurricane will be forthcoming as soon as my web-host gets their ImageMagick configuration set up correctly :-/.