iTunes Export 2.1.1 Released

With over 4,500 downloads of the GUI version, and 1,700 downloads of the console version, the 2.0 release of iTunes Export appears to be a success. Thank you to all who have helped out.

This release provides a few bug fixes and incremental improvements based on the feedback received so far. Thank you to everyone who has helped out, and please reach out if you have issues or suggestions.

This release as a few fixes for both versions, including:
  • iTunes uses different values for 'Type' based on the user's language. Changed logic to match file extension instead of value of 'Type' for MP3 file matching.
  • Updated processing of Playlist Name. All characters are replaced with an underscore (_) except A-Z,a-z,0-9,.[]{}-$=. This should resolve any issues with playlists containing illegal characters. This may result in some characters in non-western character sets to be replaced with _. GUI and Console versions now contain the exact same logic.
And a few changes to the GUI version:
  • Resolved issue with File Separator replacement (/ \) on Windows.
  • Fixed issue where scroll bars would appear with long Music Path.
  • Fixed Show Songs feature to update the song list when a different playlist is selected.
  • Implemented basic 'type to find' logic for Playlist list. Only matches first character, and you must select an item before the search works.
The new version can be downloaded from the project homepage: http://www.ericdaugherty.com/dev/itunesexport/

If you find any issues or have questions please email me (eric@ericdaugherty.com). Please include which application and version (GUI or Console, 2.1.1 etc.) of iTunes Export and the operating system you are using.

Scala and ActionScript

Like many experienced Java programmers, I've played with several other languages and platforms over time. While most development languages are not game changing on their own, they can have an impact on the speed and level of enjoyment of a developer. Development languages are like a carpenter's tools. Most tools won't change they house an experienced carpenter can build, but they may make the experience more enjoyable.

With that caveat in mind, I wanted to explore my (current) preferred toolchain, Scala and ActionScript (Flex). Not only are both languages powerful and enjoyable in their own right, their similar syntax and easy integration provide a powerful combination.

Scala is:
"... a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages, enabling Java and other programmers to be more productive."
Scala brings the benefits of a functional language, the accessibility of an object-oriented language and the power of the Java platform. I find the concise yet familiar syntax a joy to work with.

ActionScript is a dynamic language based on ECMAScript used to develop Flex applications. ActionScript brings the familiarity and dynamic nature of JavaScript, with the addition of more object-oriented features, and a powerful Flex API.

While there are some major differences in these languages, their basic syntax is very similar. To define a 'POJO' (where J is really Scala or ActionScript):

ActionScript:
package com.ericdaugherty.sample
{
public class Person
{

public var firstName:String;
public var lastName:String;
public var middleInitial:String;
}
}
Scala:
package com.ericdaugherty.sample

class Person {

var firstName:String = ""
var lastName:String = ""
var middleInitial:String = ""
}
While they are very similar, there are a few differences:

  • Scala defaults properties and functions to public, so you save on some boilerplate code.

  • In Scala, every variable must be initialized, or the compiler will assume it is abstract.

  • While this example provided explicit types, they can be inferred by the compiler.

In this example, the properties are still all statically typed Strings, but the explicit definition is unnecessary:
package com.ericdaugherty.sample

class Person {

var firstName = ""
var lastName = ""
var middleInitial = ""
}
You can leave off the types in ActionScript as well. However, this results in an untyped variable instead of using type inference to infer the static type. It also results in the following compiler warning (using Flex Builder):
1008: variable 'firstName' has no type declaration.
Finally, Flex Builder does not support code completion without explicit typing, so in practice ActionScript really requires explicit type definition.

In both languages, you can define explicit get/set behavior instead of a generic property. To only define a setter in ActionScript:
private var _firstName:String;

public function set firstName(firstName:String) {
_firstName = firstName.toLowerCase();
}
In Scala:
var _firstName = ""

def firstName_=(firstName:String) {
_firstName = firstName
}

In Scala and ActionScript, the trailing semicolons are optional, reducing the need for superfluous characters.

While these comparisons are very superficial, they do make context switching between Scala and ActionScript a bit easer. However, it is important to understand the differences in implementation. Sometimes similar syntaxes can disguise vast differences.

Note: While this example makes use of 'var' definitions in Scala, in practice a functional program would use the final 'val' definitions instead. One of my most common muscle memory issues is typing val in ActionScript instead of var.

DSL Woes

I've been struggling with my DSL connection for a while now. Over the past week it became intolerable, and I dug in to investigate this weekend.

I had been experiencing intermittent outages, which I wrote off as bad service (Ameritch and I have had issues in the past*). Over the past week I started experiencing significant packet loss (>30% on pings to the ISP's DNS server), so I knew it was getting worse.

Doing some basic troubleshooting, I thought I heard 'noise' on the line, so I replaced the DSL filter I setup. No luck.

I gave up and called SBC (Ameritech). They immediately acknowledged that something wasn't right, and referred me to their troubleshooting center. The next morning I got an automated call that I needed to call them to discuss. After some debugging on the phone, they decided that it was probably my DSL Modem. Since it was > 5 years old they thought it was probably starting to flake. I went out and bought a new one.

The new modems actually have a nice web interface so you can see what is going on. Unfortunately, it told me that my downstream connection was ~ 500 Kbps, while my upstream was 640 Kbps. Yes, this is WRONG. The downstream should have been 6 Mbps, or ~6,000Kbps. I called them back, and they agreed it was still wrong, and dispatched a technician, with the warning that there may be a charge if the problem is in my internal wiring.

I decided to verify that the issue was external, so I took my laptop, DSL Modem, and some wires outside and plugged them directly into the telephone box on the outside of my house. To my surprise it connected at full speed, indicating that the issue really was internal.

I tried some simple debugging (unplugging devices, etc.), but eventually just ran a fresh line from the inside junction box to my DSL modem. Success!

I'm a little annoyed I couldn't find the problem, but there must be some type of interference on the wiring inside the house. However, since it is all effectively hard wired together, it is nearly impossible (well, given a reasonable amount of effort) to track down. So, I'll have to settle for good enough.


* 'Back in the day' I had a frustrating experience with Ameritch (or whoever they were back then). There was noise on our line, causing my modem all sorts of problems. What really annoyed me was when they told me they only supported speeds up to 9600 bps, and if I was using a faster modem I was simply out of luck. It took a while but I convinced them that the issue was bad enough I could hear with my own ears. They ended up dispatching a truck and found that there was 'water in the line' on their end.

iTunes Export 2.1 Released

This release of iTunes Export provides some additional features requested after the 2.0 release. These features include:

The ability to override the file path separator used. This is useful when you will be using the playlist on a different operating system.

iTunes 9 File Structure. iTunes changed the file structure starting with version 9, with a separate 'Music' folder under the iTunes Media folder. This version now includes the Music folder as part of the musicFolder, excluding it from the path when using file copy with the iTunes Structure.

The GUI version now remembers your last settings and will use those as the new defaults. There are buttons to restore the initial defaults if you wish to revert to the normal defaults.

The new version can be downloaded from the project homepage: http://www.ericdaugherty.com/dev/itunesexport/

If you find any issues or have questions please email me (eric@ericdaugherty.com). Please include which application and version (GUI or Console, 2.1 etc.) of iTunes Export and the operating system you are using.

Embedding PHP in HTML files and GoDaddy Statistics

I host my site at GoDaddy, for better or worse. The site is comprised entirely of 'static' HTML pages at GoDaddy. The Blog is managed using Blogger, but it publishes HTML files via FTP to GoDaddy, and the rest of the content is HTML files I manage directly.

I wanted to avoid the redundant header/footer/etc. structures in my pages since I was editing them manually. So, I setup my site to parse the HTML files as PHP files, and then used PHP includes:
<?php include 'head.php' ?>
to build the pages on the fly. This allows me to maintain each page using simple HTML and avoid duplication of boilerplate HTML. This works because I added the following line to the .htaccess file:
AddHandler x-httpd-php .html .htm
This solutions works well, but unfortunately destroys GoDaddy's web statistics (/stats/) output. Apache attempts to parse these html files as well, but because of how they setup permissions (you have none), it fails and results in an HTTP 500 error.

This would be a simple fix if I could edit the .htaccess file in the stats directory. I could simply override the .html handler back to the default, and avoid the PHP processing.

Instead, it looks like my options are:
  • FTP the files to my local machine and view them there.
  • Create a script that copies them to a different directory (with the correct permissions and .htaccess override) and view them there.
  • Change my files to .htm or another extension that doesn't conflict with the /stats files (.html).
  • Get Creative.
Changing the file extensions isn't really an option for me as the entire point of mapping the .html files as PHP is to maintain all the existing incoming links without doing tons of rewrites. I view that stats infrequently enough that the first option is workable, but it is annoying. So I got creative...

My hack was simply to create a PHP file that reads the files out of the /stats directory. Here is a crude version:
<?php
$file=fopen("stats/INDEX.html","r");
while(!feof($file))
{
echo fgets($file);
}
fclose($file);
?>
You could expand this to be dynamically driven from a request parameter. Again, this is a HACK, but for the specific purpose it works. It should of course be secured if you want to maintain the security defined in the original /stats directory.

Flex: Monospace Fonts and Sizing Text Components

For a recent project I wanted to simulate an 80x24 terminal window in a Flex application. To accomplish this I needed a fixed width (monospace) font and the ability to size the text control for the current font. Here is what I did:

First, I needed a monospace (fixed width) font. You can embed your own fonts in a Flash SWF, but that was overkill for this effort. You could specify a specific font, such as 'Courier New' that is monospace. However, there can be issues if the client's device does not have the specific font face. Device Fonts often serve as a fallback font if the specific font requested does not exist on the user's device. In this case, I didn't want to select a specific font, I just wanted the default monospace font. So I used the _typewriter font (the other types are _sans and _serif).
<mx:text id="myText" fontfamily="_typewriter" />
Once I had a monospace font, I needed to determine the proper size of the control for an 80x24 terminal. I started by simply setting the text control to an 80 character wide (and then 24 character high) string and noting the size, but I needed something that would work easily at runtime. I created this quick helper method that would resize my control:
private function setSize(control:mx.controls.Label, charWidth:int, charHeight:int, defaultText:String = "") : void {

var testString:String = "0";
var width:int = control.width + (control.measureText(testString).width * charWidth);
var height:int = control.height + (control.measureText(testString).height * charHeight);

control.text = defaultText;
control.width = width;
control.height = height;
}
I set this method to be called when the container's creationComplete event fired. This method assumes that the control is set to an empty string initially, and accepts an optional defaultText parameter if you wish to initialize it.

The measureText method returns the size of the text, but not the surrounding control borders, so I added the default control size to the text size to achieve the proper control size.

iTunes Export 2.0 - Now Supports OS X!

This release of iTunes Export marks a milestone and major transition of the development focus. The prior releases, 1.0-1.6 were all build using the Microsoft .Net Framework, which limited deployment to Windows. With this release, iTunes Export now fully support Windows and Mac OS X.

The Graphical User Interface version has been ported to Adobe Flex and is deployed as an AIR desktop application. The Command Line version is an update of the Scala version I developed earlier this year.

iTunes Export exports your iTunes playlists as M3U, WPL, ZPL, and MPL files, allowing you to setup playlists in iTunes and use them with other software or devices.

The primary focus of this release was to enable Mac OS X support. However, in addition to this, the copy files (songs) logic has been revamped. There are now three options:
- FLAT - Copies all the songs into the output directory
- ITUNES - Retains the iTunes structure (Artist/Album)
- PLAYLIST - The 'original' copy logic that copies the songs into a directory per playlist.

The new version can be downloaded from the project homepage: http://www.ericdaugherty.com/dev/itunesexport/

As this is a major new release, please keep an eye out for any bugs or issues that may appear. If you find any issues or have questions please email me (eric@ericdaugherty.com). Please include which application and version (GUI or Console, 2.0 etc.) of iTunes Export and which Operating System version.

Closing Time

I'm reminded of the Semisonic song Closing Time today, and the quote "Every new beginning comes from some other beginning's end".

Today was my last day with my current employer, and I will soon begin not only a new job but also the process of relocating to Colorado. We're not moving right away, but today is a major milestone.

It has been fun, and I will miss many aspects of my experience here, but my family and I are excited about the future.

But on this day, it is important to keep perspective. This new beginning comes from some other beginning's end.

Google Disrupts Another Industry

Google announced plans to release a turn-by-turn GPS Navigation application for the new Motorola Android 2.0 (Droid) phone. This is significant enough, but CNET also reports:
However, Google is working with Apple on bringing it to the iPhone, and it's not ruling out licensing the software to makers of portable navigation devices used in cars throughout the world, said Gundotra, vice president of engineering at Google for mobile and developers.
This could be a major disruptive force to Tom Tom, Garmin, and the entire mobile navigation industry. There is nothing like a free alternative to a $99 iPhone Application (US Version) to shake things up, let alone the impact on the automotive manufacturers and their in-dash software needs.

Microsoft and Apple are obviously concerned about Google, but Google's impact may be greater in less obvious market areas.

Signing AIR Applications: How AIR's Certificate Requirement Complicates Open Source Development

Adobe AIR is the desktop runtime environment for deploying Flex applications. The main difference between a Flex application and an AIR application is that AIR applications are installed locally and have additional access to local resources, including the local file system, while a Flex application runs in the browser and is 'sandboxed'.

I've been developing an AIR version of my iTunes Export open source application. The original version was developed in .Net, making it Windows only. The AIR version will work across Windows and OS X, and will replace the existing .Net GUI.

As I prepare to release the first Beta I ran into the question of how to sign my AIR application. All Adobe Air applications must be signed using a digital certificate. There are a few companies that sell certificates that are recognized by Windows and OS X by default, and they charge $300/year and up. That is pocket change for any company selling an application but a pretty significant cost for an open source application that produces no revenue.

You can self-sign a certificate, but you are then presented with this dialog box:



If you want to avoid this dire warning, you must buy a certificate from one of the Root Certificate Authorities.

So what do I do? I'm tempted to release it using a self-signed certificate. The warning is annoying, but I've been releasing the .Net version unsigned for years, and it had as much or more 'destructive capability'. However, the users also were not faced with this warning dialog.

It would be nice if there was an 'Open Source' Certificate Authority (CA), that allowed open source projects access to free certificates, but the costs involved in becoming a Root CA and managing the issuance of certificates would require a very generous patron.

If you are looking for step by step instructions on HOWTO sign an AIR application, check out this tutorial.

Parsing PList files in ActionScript

The PList format is not one of my favorites. I've complained about it before, but Apple continues to use it for its iTunes library information, so I continue to parse it.

I've been exploring the option of porting the iTunesExport project to Flex (Air) to provide a cross platform version. The current version is written in .Net and therefore not usable on OS X. As such I needed to parse the PList file using ActionScript. I adopted the Java PList Parser written by Christoffer Lerno for his XMLWise project.

The dynamic nature of ActionScript resulted in a much smaller implementation (although I did skip over 'REAL' and 'DATA' types), and makes it easy to consume.

You can view or download my version here, released under the MIT license.

Here is a snapshot of how it is used:
var parser:PListParser = new PListParser();
var pList:Object = parser.parsePList(xml);

playlists = pList.Playlists.filter(filterPlaylists);
The last line accesses the Playlist array (dynamically created when the PList is parsed), and filters it using the specified filterPlaylists method, returning a filtered list of the playlists from the original XML file.

It provides easy structured access to an XML file that would otherwise be difficult to access in a rational manner.

iTunes Export 1.6 Released

A new version of my iTunes Export utility is available.

iTunes Export exports your iTunes playlists as M3U, WPL, ZPL, and now also MPL files, allowing you to setup playlists in iTunes and use them with other software or devices.

This release adds several fixes and enhancements that have build up in the last few months. They include:

Improved the logic for music copying. It can now handle duplicate songs and an arbitrary number of songs in a playlist (removed the 999 song limit) and handles duplicate files gracefully. Thanks to Simon Hyde for submitting the patch.

Added support for MPL playlists, used by Centrafuse (www.centrafuse.com) Car PC. Thanks for Stavros Glezakos for submitting the patch.

The new version can be downloaded from the project homepage: http://www.ericdaugherty.com/dev/itunesexport/

There have been several requests for improved file copying approaches to handle the many different usage scenarios. I have begun work on a 2.0 version and am planning to include a revamped approach to file copying in that release. Thanks for all the feedback.

Please email me with new features or bug fixes (eric@ericdaugherty.com).

Java Email Server 2.0 Beta 2 Released

Java Email Server (JES) is an open source email server (SMTP/POP3) written in Java.

This release is the second Beta version of the new 2.0 development branch. This is an incremental update to Beta 1 and contains the following updates:
  • Truly decoupled dependency on log4j. See section "Logging Facility" in AdditionalNotes.txt.
  • Server instance now uses a singleton pattern.
  • Added an option ("testing.destination") that, when set, directs all outgoing messages to the destination (currently a folder).
  • Each domain gets its own default user.
  • A profile can now be set for the services to be activated at startup. The profile is broken down into Mail Transfer Mode and Mail Retrieval Mode. Resource allocation is affected, but not considerably.
Please read the Beta 1 Post for the other 2.0 branch changes.

While the belief is that JES 2.0 Beta 2 is stable, we will continue with Beta releases in the 2.0 branch until we feel confident that the 2.0 code is stable and production safe. Please provide feedback on this release in the JES Google Group, even if it is just letting us know you are using JES without any issues.

You can download this release from the project home page.

Eating Crow

After holding out for years, and picking the Palm Pre as my 'next phone', it is time for me to Eat Crow.

Last weak I broke down and bought a brand new black iPhone 3GS. And I love it.

Two years ago, when the first iPhone came out, I wrote the post 'No iPhone for Me'. Re-reading that post now, I think my criticisms were mostly fair, and have been mostly resolved. My primary issues were:
  • Price ($499 and $599)
  • No Removable Battery
  • Touchscreen Keyboard
  • EDGE (no 3G)
I wrapped up the post with the following: "However, even if the iPhone's first version does fail, it is hardly down for the count. Apple can and will innovate, and with their existing iPod market domination, they will have several chances to make this concept successful. I'll be tracking how this develops (from my Treo)." Apple did use subsequent releases (OS and Hardware) to make it successful. The addition of 3G and the price drops were major improvements, and directly addressed half of my concerns. I've come to accept the keyboard as an acceptable option. I'm not convinced about the built in battery, but it is clearly the direction Apple is moving, with their laptops now as well. And the built in battery provides a fit and finish that is impossible with removable battery devices. But it was the App Store, with all its drama, that made the device a HUGE success.

As I become more and more fed up with my Treo, I explored my upgrade options. There was the Palm Pre, Blackberry Devices, Android, and Windows Mobile. I'd tried Windows Mobile and had no desire to go back. Blackberries also held no allure. They are very business focused, and the iPhone and Pre both provide better balance. In the end, it was the market share (Apps, general mind share) that convinced my to choose the iPhone. My wife has one and loves it, and while I think the Pre is a great device, I'm not willing to make a 2 year (wireless contract) bet on it.

After a week of use, here are my thoughts:
  • The touch keyboard works. I don't LOVE it, but it works.
  • The navigation is less efficient than my Treo, but it is a small trade-off given the other benefits.
  • The integration of the iPod functionality is great.
  • The Exchange support is great.
  • The battery is OK.
  • Safari is great. I'm actually using Safari for GMail instead of the Mail app for now.
  • Google Reader on Safari is great.
  • Apps, Apps, Apps.
I realize I'm a late comer to the party, but I'm going with the idea that I waited until the iPhone was ready for me. Either way, I'm glad I made the switch.

Timeless Songs

There are great songs, and then there are timeless songs. Songs that are appreciated generation after generation, and throughout the different stages on one's life.

I think part of the test of a timeless song is how an acoustic, or 'unplugged' version holds up. When you strip out all the energy and 'noise', do you still have something that you love?

I just saw an old ('75) acoustic performance of Thunder Road (thanks Merlin), and it reminded me of why the song is so great. When you strip it down to its core, it is still amazing.

This song and album are both available as MP3 downloads at Amazon (affiliate link).

MTV Unplugged popularized the idea of acoustic versions of rock (and other genres) songs. As a Bon Jovi fan, I've enjoyed a couple Unplugged versions of Living on a Prayer, but the classic one is the '89 MTV Awards:



Acoustic versions often give you a closer and more intimate connection with the lyrics. While this is generally true, and certainly true of the previous two songs, I don't find this Dylan Unplugged version of Like a Rolling Stone creates that closer connection. I think it is too 'busy'. To many instruments, too much noise. Take it as a contrast to the previous two. It is still a great song, but I much prefer the more distilled versions by Bruce and Jon.

Building Flex Applications with Gradle

In a recent post, I touched on my frustration with Maven and my interest in other build tools. Gradle looked interesting (Grovy DSL with Ant integration). I've been playing with building a Flex app, so I thought I would see if I could get Gradle to build it. Since Adobe provides custom Ant tasks as part of its SDK distribution, it should be easy. In fact, it is.

Assumptions:
  • You have downloaded the Adobe SDK (or Flex Developer)
  • You have a sample Flex (or Air) app to build.
  • You have Gradle downloaded and installed.
The first step is to create a Gradle build file (build.gradle) in your Flex application directory. To test your Gradle install, put a simple Hello World task:

task hello << {
println "Hello World"
}

You can then run gradle hello from your command prompt. It should echo back:

:hello
Hello World

Now you need to 'install' the Flex Ant Task Jar. This can be found in your Flex SDK Install under the ant directory (sdk_install/ant). There are a couple ways to install this. The easiest is to put it in the lib directory under your GRADLE_HOME directory. You can also specify the classpath in the taskdef command, but I'll leave that as an exercise for the reader...

Once you have the Flex Ant Tasks installed, you can create your Gradle build script. Here is a straight foward compile script:

ant.FLEX_HOME="c:/Program Files/Adobe/flex_sdk_3"

ant.taskdef(resource: "flexTasks.tasks")

task compile << {
ant.mxmlc(file: "src/Main.mxml") {
}
}

You will need to change the FLEX_HOME value and Main.mxml to your specific environment/app, but that is it. You can then execute gradle compile from your command prompt and compile the application.

If it is an Air app, you will need to add configname: "air" to the mxmlc task definition so it uses the Air configuration. Otherwise, you will likely see: Unable to locate specified base class 'mx.core.WindowedApplication'

To build the Air package for your application, you will also need to call the script file included with the SDK. You can find a sample of the ANT version of this here.

This example is more a sample of how easily Ant tasks can be used in Gradle than the overall power of Gradle. But it does demonstrate that Gradle can be used to build anything that has Ant tasks, and that is a great start.

Build Tools

I've said it before, I find Maven annoying. I've even explored other build tools like Simple-Build-Tool, which I like, but I'm not sure I love.

Now Steve Ebersole from Hibernate joins the ranks of the frustrated Maven users. He is frustrated with Maven's module setup, something I've seen first hand on my current project.

Based on the comments on his post, it seems that Gradle is becoming a popular alternative, I'll have to check it out.

Simple-Build-Tool is also mentioned, though I've only used it for Scala projects so far, I find it interesting but also a bit difficult to fully understand. It's flexibility and use of Scala is powerful, but has the common side-effect that it isn't obvious how to accomplish something, or what a configuration line really does. I like the fact that it leverages Maven's standard structure, allowing you to easily test Maven projects with SBT.

I think Ant could also be salvaged, if it adopted the Maven directory conventions and then allowed configuration from there, but I'm not sure that it the best approach. It may just be time to move on.

What do I really want? A tool that provides simple setup for 'standard' projects, but allows the flexibility and power for fine grained control when I want to devaite from the 'norm'.

Deceptive Marketing

There is a great post on 37 Signals about 'Little Green Lies'. I've looked at the exact same label ('98% naturally derived ingredients') and thought the same thing.

What does that even mean? EVERYTHING is 'derived' from something natural at some level. The sad part is that marketing like this works.

John Hughes

With John Hughes' passing yesterday, it is appropriate to pay tribute to his life and work.

For me, he was the writer and director of the movies that defined my adolescence. As a writer, he had an amazing string of hits from 83-87 with:
To write that many great movies every year for 5 years is amazing, especially since he directed over half of them himself. I've seen every one of these movies many times and enjoyed them all. Add other hits like The Great Outdoors (1988) Uncle Buck (1989), National Lampoon's Christmas Vacation (1989), and Home Alone (1990), and it is an amazing career.

Growing up in Chicago, John Hughes' movies always had a special appeal. They were often set in Chicago or its suburbs, and represented Midwestern suburban kids. Me.

Here is a great montage on YouTube set to a great song...


Mobile Web

Fred Wilson has an interesting post about the 'Mobile Web', and how it is different than the world wide web we know. In short, the mobile web is highly fragmented. Read the article.

This is an issue that has impacted my thought process recently. I've been considering getting a new mobile phone (PDA), and as a long time Palm user I'm interested in the Pre. The OS looks great, it has a physical keyboard, and the device is fairly open (you get root access with the developer SDK). However, in the fragmented mobile market, the iPhone is the king. The development effort and ecosystem around the iPhone is amazing. In a fragmented market, I'm not sure I want to bet on a niche player.

There are many reasons for this fragmentation, some technical, some institutional. I'd like to see more of the institutional ones removed (no exclusive carrier agreements, more 'open' devices), but on 'limited' devices, tailoring the experience to the hardware is important for usability. There is a reason that the iPhone SDK was the key to the devices explosion. Apps delivered in the iPhone browser are just not as compelling.

To achieve a mobile web as ubiquitous as the world wide web, we need to make the browser the default application delivery platform, just as it is on 'real' computers. I'm just not sure the devices and networks are there yet.

Sports, TiVo, and Niche Markets

I enjoy watching sports. Pro Football, College Basketball, and all types of Soccer. I also enjoy my DVR (not technically a TiVo anymore). Unlike many people, I enjoy watching sports on my DVR. I'm not hung up on 'live', and I really dislike commercials. I don't particularly care for halftime either.

When I watch Pro Football, I have to watch the game on the same day. I don't really ever watch it live, but usually I watch it within a few hours of 'live'. It is pretty difficult to avoid finding out the end result of the game for more than a few hours, and especially on Monday mornings in an office environment. College Basketball is better, but still tough to delay more than few hours.

Soccer on the other hand, is a niche market. Last weekend I recorded 4 games. The Chicago Fire, Chicago Red Stars, and the Men's and Women's national teams. I watched the fire game and the men's national team game over the weekend. The Red Stars and Women's game I watched during the week. I never heard the scores on the radio, no one twittered about it, and there was no water cooler talk. I am free to delay the games as long as I want. In fact, I've only heard scores from a Red Stars game once on the radio, on a really slow sports day. Even the WNBA gets more time than women's soccer. Does anyone really care about the WNBA?

I don't watch ANY television live. What's the point? I let my life be my life, and television provides entertainment when I'm ready. I'm glad the sport I enjoy most supports it.

The Simple Approach

I have a reasonable amount of archived email on my local hard drive, that I'd really prefer to store in Gmail. Since I use Google Apps for my domain I created a new account to store the archive email.

The next step was figuring out how I could get the email into the account. I started looking around for applications to export to Gmail or import from archive formats. Then I came across the simple approach.

I added the newly created GMail account to my email client that contained the archived email. Then I dragged the email from the archive folders into the new account. The email client dutifully copied it to the Gmail account, preserving the date and all other meta-data.

No tools needed. Simple, easy, and effective.

Vacation

After a two week vacation, I'm back home.

It was a great trip, and very relaxing. My family and I spent the time in Colorado exploring the Rockies and enjoying the peace and quiet. We hiked to the top of Flat Top, Twin Sisters, and Estes Cone, along with a few other great non-summit hikes. Here are some of my favorite shots from the trip...

In Golden we watched kayaks doing jumps in the river.


Calypso Cascades


An HDR image of Estes Park from Little Valley.


The coming storm, taken from the top of Twin Sisters.


Fireworks over Estes. This was the pre-game show.


Fireworks over Estes. This was the official show.


The hummingbird feader was popular. We refilled it 3 times while we were there.

Kayaks in Golden

I went for a walk with the family tonight in Golden, CO and came across some kayakers doing jumps in the river. My shutter speeds are a bit slow, but still some cool moves...

Edge Cases

In tools like Outlook, there are an impossible number of ways users can interact with the tool. This creates lots of 'edge cases' that don't get tested or developed for.

Here is one:
  1. Use the default setting that does not have the BCC line shown.
  2. Send an email with BCCs by clicking on the To: button and adding entries to the BCC line in the dialog box.
  3. Send the message
  4. Realize you want to reuse the body in a new message.
  5. Open the sent message and click resend.
  6. Type in a new To: address
  7. Send it to everyone you previously BCC'ed because Outlook doesn't show you the BCC line, even though it has entries.
Oops.

This is an edge case where adding a user to the BCC line causes it to be displayed, but opening/resending an email with users in the BCC line DOES NOT cause it to be displayed. The trigger logic is flawed, but you don't realize it is flawed unless you are aware of other ways that a specific state can occur. In this case, the logic to display the BCC line should be triggered by the 'state' of having entries in the BCC line, not the event of adding an address to the BCC line.

Netflix Saturday Processing

I was surprised to get an email from Netflix today notifying me that a new disc shipped out. Netflix does not normally process movies on Saturday, so mailing a movie back on Friday means you have to wait until Tuesday to get the replacement.

I did a quick search, and indeed, it looks like Netflix is introducing Saturday processing across the US.

A great service just got better.

Google Wave

Google recently announced a new collaboration platform, and I finally got around to watching the video.

Wow.

I started watching it as a small video in the background and kept finding myself switching to full screen so I could really watch and follow what they were doing. I'm not going to attempt to summarize the 80 minute video introduction, nothing I could say would really cover it. You should spend the time and go check this out.

I will talk about some of my reactions:

First, wow. This is a solution to the fragmentation of communication. Email, IM, Twitter, Facebook, etc. There are too many channels we use to communicate, many of which are dated.

Open Platform, Open Source (mostly).

Build on GWT. HTML 5 seems to be a big focus.

RIAs - Sun, Adobe, and Microsoft are pushing their rich client platforms. Apple and Google are focused on HTML 5. This app makes a pretty strong case that HTML 5 is sufficient for nearly every app.

Playback - a cool concept that provides context that may be lost along the way. Integrates with the plugins, etc. Cool.

Convergence - Google has launched quite a few apps of varying success. Mail, Maps, Apps, Social Networking, News, Translation, etc. Wave takes these and combine them in a way that far exceeds their individual value.

Extensions - This is an extensible platform of course. Wave is cool today, but will be much more tomorrow.

Federation - This is what makes the entire concept feasible. You can have corporate wave servers but still interact with the various vendors, consultants, etc when necessary, and everything that is private never leaves your server.

Wow.

Mac Window Managment

My wife just bought a new 13" MacBook Pro, and as longtime windows users, we're both working through learning a new system. Overall she's pretty happy, but we just spent 15 minutes today figuring out an issue with windowing...

It started when I migrated over her iTunes library from her Windows box. The great thing was that not only did it work great (binary compatibility), but that it also auto-found the music (I had it on the M: drive on Windows and I moved it into her iTunes Music folder on the Mac as she has plenty of disk space).

However, the window defaulted to a size larger than her screen, and this is where the frustration started. On windows, this would be fine, as you can resize a window from any edge. However, on the Mac you can ONLY resize from the bottom right corner. You also can't move a window off the top of the screen. No matter what we tried, we couldn't get the bottom right corner on the screen to allow us to resize.

In most apps, this would probably be fine, as the green key would resize the window. However, in iTunes this button switches to the mimized player view.

After some Google searching, I figured out I could use the Option key with the green button, which finally worked. Not intuative or flexible.

Overall, she's still happy, and some learning curve is expected.

Palm Treo 755p and Exchange

My company upgraded/migrated our Windows Domain last weekend, and as a result, my Treo 755p's ActiveSync Support stopped working. When I entered the setup information and tried to run the initial Sync, I got the error codes: 1913 4828.

There is some confusion about what this error code means. It can often be related to SSL certificate issues, which may have been part of my problem. I found this page very helpful to debug the SSL issues: http://kb.palm.com/wps/portal/kb/common/article/16733_en.html

Once I had that working (I did have to change the server name to match our certificate) I was still having issues, and I came across this post: http://episteme.arstechnica.com/eve/forums/a/tpc/f/579009962631/m/417009683931 that suggested the issue was the ActiveSync security policy. I tracked down our IT guy and cajoled him into testing out removing the policy, since ours didn't really do anything anyway. It worked! The odd thing is that the old domain seemed to have a policy as well but something must be different.

So apparently the Treo (Versamail 3.5.5) doesn't support ActiveSync with security policies. Ya, ya, time to upgrade to a real phone...

Java Email Server 2.0 Beta 1 Released

There have been multiple efforts started over time to create a modernized version of JES (It is an 8 year old fork of an even older project) that would incorporate features more or less expected from a up-to-date MTA/MDA. I'm happy to announce that Andreas Kyrmegalos has stepped forward and developed a heavily revised and augmented version of Java Email Server that will be released as 2.0 Beta 1. Staying true to a gui-less configuration approach hasn't prevented a host of new features to be introduced.

While the changes are too numerous to cover in this announcement, I wanted to highlight a few of the more important changes:
  • TLS/SSL Support
  • Configurable sandboxing
  • Support for white/blacklisting
  • Support for spam filtering/virus checking via amavisd-new using a dual MTA approach
  • Data directories are now configurable (incoming/outgoing email storage)
  • New Service Wrapper (Tanuki Java Service Wrapper)
  • More efficient mail dispatching to multiple users at a single domain
  • Cleaner shutdown process
  • Mail transactions with mail servers employing reverseDNS checks (useful for JES instances on a dynamic IP)
  • More efficient memory handling
  • On the fly POP3/SMTP port listening switching
  • Interfaces to enable extension modules
  • Migration tool for JES 1.6.1
  • Introduction of an automated testing framework
  • Improved MIME header parsing support
  • 8BITMIME, SIZE extensions support
  • SASL MD5-DIGEST, GSS-API support
  • and MUCH MUCH more.
JES is now dependent on JDK 1.5. The existing 1.6.x branch will continue to be available to support JDK 1.4.

Project management is being carried out using Maven.

JES is also getting a new license. The existing GPL license was due to the original GPL license of CRSMail. CRSMail has now been released into the public domain, allowing JES to be re-licensed under a BSD Style license.

I am also launching a new Google Group to facilitate a more open support structure. Please post all questions about JES to this group.

This initial release is a Beta. It has been used in production environments, tested under Windows XP, Ubuntu and Windows Vista on single and multi core systems and should be solid, but it is possible that some issues may exist with specific configurations/environments. I encourage everyone to give this a try and provide feedback. The goal of this version is to provide a much needed step forward for JES while retaining the simplicity and ease of configuration and deployment. Let us know if this release achieves the goal.

The 2.0 Beta 1 release is available to download from the JES Home Page and a version 2 branch resides at sourceforge's subversion repository. Give it a spin and post your comments in the Google Group.

Java Email Server (JES) Eclipse Plugin

There has been a flurry of activity on the Java Email Server front recently. A beta release of JES 2.0 is imminent, but today I'm announcing the availability of an Eclipse Pluign for JES.

An Eclipse plugin? Isn't JES an email server? Yep. Here is the introduction from the project page:
Suppose that you developed a service in your application that send mails to clients (confirmation, cancellation, alert, etc...) and you want to test it and view the layouts of theses mails? Most probably, you will use your enterprise mail server and your professional account as test account.
But wasting long time waiting to receive test mails? connection problems? want to test cc or bcc? your professional mail account is full of test mails? this plug-in can help, it launch a local instance of JES mail server for test use, no installation is needed, easy to configure and provide accounts management (you can add as many accounts as you want).
The project, JES email server launcher Eclipse plug-in, or jesplugin-in, was written by Nabil Dridi. Nabil is a developer from Tunisia, further expanding the list of countries that have helped grow JES. Thanks to Nabil and all the others who have helped out over the years!

Nabil has done a great job putting this plugin together I and recommend it for anyone using JES as a development resource. Give it a try!

Second Life

Reading through Google Reader today I came across a post by Bruce Schneier mentioning that he would be interviewed in Second Live today.

My first thought: Second Life? Oh ya, I remember that. That still exists?

Not long ago I couldn't open up my feed reader without endless discussion of Second Life. Every company was talking about creating a virtual presence. There were discussions of an alternate currency, virtual economy, money laundering, and of course the flying penis attacks.

Fast forward two years, and my reaction to a post about second Life is 'really, that still exists'?

Things move fast on the web. What is hot today is not tomorrow (remember hot or not?). MySpace was going to take over the world. Now it is FaceBook and Twitter.

As a late comer to Twitter and one who still avoids FaceBook, I wonder how we'll view each two years from now. Granted, both have an adoption rate that I believe surpasses MySpace and Second Life, and comparing Twitter to Second Life is beyond Apples and Oranges, but if we've learned anything in the past decade, it is that nothing lasts forever (or even very long) on the web.

Fed up with my Treo 755p

I'm done.

I've been a user of Palm OS devices for over a decade now, and I'm fed up. My current phone, the Treo 755p is annoying me on a near-daily basis.

I've been a huge fan of the Palm OS for years. Their early devices were groundbreaking (I started with the Palm Personal, and loved the Palm V). They enabled me to track my personal information and provided a platform for applications (OK, games).

I loved the evolution to the Treo line. Is started with the Treo 300, and I owned the 600, 650, and now 755p. For a brief moment I had a Samsung i760 but quickly fled back to the 755p. I missed the ease of use and simplicity of the Treo form factor and UI.

But the OS and hardware line has gone on too long. My phone crashes too often, I spent 10 minutes this morning figuring out why my headset unpaired from my Treo, and I'm tired of having a sub-par web browser. And when did Palm, the king of 3rd party apps, become a developer wasteland? Yes, I know, when the iPhone was released. I even wrote my own Palm applications back in the Palm V days.

Anyway, I'm done. I need to move to a new phone. Windows Mobile phones are out of the picture. They are just too big of a departure from the usability of Palm devices that I believe is critical. The iPhone does beckon, as it has huge 3rd party support, a great browser and screen, and integrates with Outlook's Active Sync. The other option is the Palm Pre. As a long time Palm fan I'm very hopeful that the Pre will reignite the Palm brand and yield another decade of great devices. The early reviews are solid but not overwhelming. It is version 1 of a new OS. As a long time fan of a physical keyboard the Pre does seem to be an ideal solution.

I'm also under contract with Verizon, and neither of my options is available on their network today. There are rumors of the Pre at the end of the year and even rumors of an iPhone on Verizon in the future. It will be an interesting week as the Pre launches Saturday and Apple's WWDC takes place next week.

We'll see what shakes out in the next week or so, and then I'll finalize my migration plans.

Chrome vs. Firefox

Firefox is great.  I've been a fan for a long time and it was my exclusive browser for years.  However, I've recently noticed more crashes and instability and I decided to try an alternative.  I installed Chrome a while ago but it has sat mostly dormant on my drive.  A week ago I decide to make the switch.

First, this isn't really a fair comparison.  My Firefox profile has years of bloat and plugins.  Comparing this to a pristine Chrome install is certainly not Apples to Apples.  But that said...

Chrome's model of one process per tab seems to be much more solid than Firefox's single process approach.  I have not had a single crash or general 'Not Responding' issue with Chrome.  However, I have noticed that switching to tabs that I have not used recently often triggered a delay and furious disk activity.  It is a trade off, but I think the Chrome approach is preferable.

Search - I'm used to hitting the / key and typing to search within a page.  It is Ctrl-F in Chrome.  Ctrl-F is probably more consistent with other apps, but muscle memory is tough to change.

File Downloads - I'm used to using a plugin on Firefox (Download Statusbar) and I like the Firefox plugin better than the Chrome default.  Specifically I like the ability to pause, resume, and copy a download URL.  

The Chrome window itself is very googley, and by that I mean sparse. There is no status bar, only a link display when you select or hover hypertext.  There is no title bar and so far I have not added any new toolbars.  I really like the ability to move tabs into their own window or a separate window.

The one feature I miss the most is the ability to create shortcut bookmarks to search functions.  For Wikipedia and IMDB, I have shortcuts created so that I can type 'imdb start trek' in the address bar or 'wiki Star Trek' and it will take me to the search result page.  I'm not sure if you can do this in Chrome but if you can I have not found it yet.

I also find that Chrome's built in spell checker is inconsistent.  It does not seem to work at all in Blogger's edit box, but in other cases it works fine.  Obviously this is annoying.

There are a lot of similarities and it is obvious the Chrome team set out to build a 'better Firefox', as opposed to a radical new approach.  The bottom line is that both browsers work well and either could serve as my default.  For now though, I'm going to continue the Chrome experiment.

Star Trek and Caprica

In addition to catching up on yard work and playing with the kids, I spent some time this weekend catching up on my scifi movies.

I finally made it to the theater to see Star Trek. I was very impressed with the movie overall and especially with how they positioned the storyline. Without giving too much away, they setup the story to allow them to continue with this cast in future movies without endless harping about continuity and other concerns you can imagine from a rabid fan base. I thought it was brilliant. The overall movie and cast were great. It 'hit every note' it needed to, and the actors were great. Scotty was my favorite but Spock was very well cast as well. I'm excited to see the next installment. And about the lens flare... Yes, there was a bit of that. I really noticed it in the first 1/3 of the movie but after that it either toned down or I got used to it. I did like style though, even if it was a bit over done.

Caprica, the pilot for a series 'prequel' to Battlestar Galactica, was release straight to video. The series is scheduled to begin airing in 2010. The series is set 58 years before the events of BSG and depicts a pre/early-Cylon Caprica. While it takes place in the 'BSG' world there is a stark contrast between the relative 'low-tech' society of BSG and the 'shiny high tech' Caprica. The series could be interesting, but anyone expecting a show resembling BSG will be disappointed.

I'm sure it will be weeks until I make it to see the new Terminator.

Simple Build Tool

As part of my foray into Scala I came across a new build tool called Simple Build Tool (sbt). As the website states, "sbt is a simple build tool for Scala projects that aims to do the basics well."

I didn't really come across sbt, it came to me. Tim and Mark commented on my Maven vs. Ant post and I decided to check it out.

It is similar to maven in its dependency management and configuration by convention. If you do need explicit configuration you write it as actual scala code, which is interesting. Obviously it is powerful. You can do pretty much anything, but the flexibility and relative immaturity (it is version 0.4.5) of the project makes it difficult at times to determine how to do something.

sbt runs as an interactive session, so it solves the java startup overhead by just staying up. In practice this actually worked well for me, and provided the fastest compile/package times of any of the tools I used.

For the issues and questions I did have, the support in the google group was great. Mark especially helped me track down and solve questions and issues I had. Some were my own learning curve, some resulted in patches to sbt. The last issues we worked through was making an executable jar, which turned out to be a platform issue (/ versus \) that is getting patched.

If you are working with Scala I encourage you to give it a spin. I think it has a future as it continues to develop and gain users.

iTunes Export Scala 0.2 Released

I released a new version of my iTunes Export Scala utility.

This is a port of the original .Net application to Scala. Scala is a hybrid Object Oriented/Functional language that compiles to Java class files and executes on the JVM. This port enables Mac OS X users to access iTunes Export features. The original iTunes Export application was written in .Net and does not run on Mac OS X.

iTunes Export exports your iTunes playlists as M3U or WPL files, allowing you to setup playlists in iTunes and use them with other software or devices.

This release adds support for Extended M3U, WPL, and ZPL playlist formats. It also provides the ability to override the default (or arbitrary) path prefix for the music files.

The project home page is here: http://www.ericdaugherty.com/dev/itunesexport/scala/

The project is hosted at Google Code. You can check out the Google Code project home page if you want to browse the source tree or track issues, etc.

I have received several new feature requests and bug reports recently and I plan on getting to them soon. Don't be afraid to ask for new features (eric@ericdaugherty.com).

Google Code

I've always hosted my open source applications at SourceForge. When I first started developing open source applications, it was the de facto standard. When it came time to find a host for my new iTunesExport-Scala application I decided to check out the landscape. The primary sites I see these days are Google Code and GitHub. Since I have not yet embraced Git I decided to give Google Code a try.

What do I think? I like it. All the Google applications seem to have a common theme: they are simple to use and cover the core 80% well. In fact, their approach is a lot like Apple's approach. Google is just an uglier Apple. :)

Setting up Subversion, doing releases, and editing the home page were all much easier than on SourceForge (even after their redesign). The site is fast, and integrates with Google Analytics so I can track traffic in the same tool I use for my primary website.

The bottom line: It just works.

iTunes Export Scala 0.1 Released

I am launching a new project: iTunesExport Scala. This is a port of the original .Net application to Scala. Scala is a hybrid Object Oriented/Functional language that compiles to Java class files and executes on the JVM.

I am launching this port to enable Mac OS X users to access iTunes Export features. The original iTunes Export application was written in .Net and does not run on Mac OS X.

The project home page is here: http://www.ericdaugherty.com/dev/itunesexport/scala/

The project is hosted at Google Code. You can check out the Google Code project home page if you want to browse the source tree or track issues, etc.

This is my first Scala application so I would appreciate any feedback on the source code. My goal is to continue to add features until parity is reached between this and the original.

Maven versus Ant

The Java world has two popular build tools, Ant and Maven. I've been using Ant for as long as I can remember. Maven is a newer tool aimed at addressing some of the pains of Ant and also providing an more complete experience.

For a long time I avoided Maven. I tried out Maven 1.0 early on and was frustrated with the lack of control and underdeveloped eco-system. With its dependency managment system, dealing with dependencies that are not in public repositories is annoying, and early on most libraries didn't have versions in the public repositories.

I'm working on a Scala port of my iTunes Export application and I had to decide how I wanted to build the project. Here are the pros/cons I saw:

Ant
  • Well Known - I know how it works, and there are tons of examples
  • Malleable - I can make it do what I want. I never have to fight the tool to make it work the way i want.
Maven
  • Less boilerplate code - Maven provide you many features, like compling and assembling packages 'for free'.
  • Dependency Management - Project dependencies are automatically downloaded from public repositories and your releases can be uploaded to public repositories for distribution.
  • Its the new Black - It is the current 'standard'.
For iTunesExport Scala I'm using Ant, although I am providing a Maven pom file as well for developers that prefer Maven. The basic Maven project was easy to setup, but I found myself struggling to make other things work, like unit testing (It doesn't work out of the box with ScalaTest) and packaging. In Ant, I already have predefined targets for most of what I want, and it takes just a few seconds to tweak to my exact desires.

For larger projects, I think Maven is worthwhile. The dependency managment is important as projects grow and the Maven ecosystem is large and growing. For small projects, I'm sticking with Ant.

iTunes XML Parsing - .Net vs. Scala - UPDATE

In an earlier post I compared a .Net and Scala implementation of an iTunes Music Library XML parser. The Scala version took a long time to 'load the XML file from disk'. Since then I realized that the default behavior of the parsers is to validate the Schema. I had disabled this behavior in the .Net version but not the Scala/Java version. Much of the 3 seconds to 'load' the XML is really an external HTTP request to download the xsd file and perform the validation. However, using the Scala library I do not see an easy way to disable the Schema validation (Under the covers the Scala library is delegating to existing Java APIs).

Oracle buys Sun?

Oracle and Sun announced an agreement for Oracle to acquire Sun. What do I think?

It's better than IBM. And it is aimed at IBM.

IBM and Sun seemed to have a lot of overlap. Certainly IBM has a well defined hardware business, and acquiring Sun's hardware business doesn't really help. IBM is heavily invested in Java so I'm sure they would have enjoyed more influence over its growth and direction, but it didn't really seem compelling.

Oracle on the other hand puts the final piece of the puzzle together. Oracle is now a credible Enterprise IT partner. With the addition of Sun's hardware business, Oracle can compete with IBM at nearly every level. If I'm a large company, I can turn to Oracle for:

Storage Solutions
Servers
Database
Enterprise Software Packages
Custom Enterprise Software Development (Java, WebLogic, etc.)

If you are a large company, you can either hand your checkbook over to Oracle or IBM. Or Microsoft+HP.

As a user of Java and MySQL, I'm hopeful that they will at least continue on as is and hopefully improve. I don't expect any real changes in either for a while though.

iTunes XML Parsing - .Net vs. Scala

As part of my iTunesExport utility I wrote a (C#.Net) module to parse the iTunes XML file and provide two simple collections, Playlists and Tracks. As part of my effort to learn Scala, I rewrote this module in Scala. While the libraries don't provide the exact same interface and functinality, they are effectively the same. The stats:

(Physical) Line of Code count:

.Net: 459
Scala: 226

The Scala version accomplished the same functionality in 1/2 the lines of code. While LOC count is a bit arbitrary, I think it is an important point. The Scala code is much more concise, but maintains or even improves the readability (if you have a reasonable familiarity with Scala). In this specific case Scala's handling of public properties provided a big reduction. Ex:

Scala:

val trackId: Double = ...
.Net:

private string id;

public string Id
{
get{ return id; }
}
While it may seem that this provides much of the savings, the Scala version provides a much more verbose toString method, so some of the per-property savings is actually understated in this comparison.

While developing I did find myself consistently following the pattern:
  • Identify logical functionality I wanted to extract into a helper method
  • Writing code for helper method
  • Realize it was just a single line and moving back inline
Simply put, I found the structure and features of the Scala language very well suited to write high level understandable code that performed the same functionality in fewer lines of code, compared to C# or Java.

Performance was something else. As a functional language Scala is well suited to be a scalable language. It also runs on the JVM, benefiting from a significant amount of work building a high performance virtual environment. That said, this implemenation was purely single threaded. The initial results were somewhat surprising:

Scala 4,800 ms
.Net 650 ms

That seemed like a pretty significant difference, so I took a deeper look. The Scala library to load the XML file from disk took nearly 3 seconds! This is a huge hit and accounts for much of the performance difference. The remining functionality took ~1,800 ms, nearly 3x the entire .Net solution. Where does the time go?

~3000 ms - reading the XML file from disk using Scala's XML library
~100 ms - parsing the main library attributes.
~1700 ms - parsing the tracks
~60 ms - parsing the playlists

Clearly loading the file and parsing the tracks are the biggest targets. I'll leave loading the XML out of scope as it is a Scala library and focus on the tracks. For each of entities (Library, Playlist, Track) I use a Trait that parses the plist XML into a Map and then assign the properties from the map. For each track, parsing the XML into a Map took 1 ms or less, but assigning the variables from the map (and doing any type conversions) took up to 4 ms but more commonly took 1 ms or less. At the single ms level the simple act of measuring introduces a significant impact, so I'm not sure these numbers are meaningful.

With over 2600 tracks in the library, even spending 1 ms per track is forever (and indeed the average time per track without the measurement was closer to .65 ms). The additional complexity introduced by my Scala approach caused a significantly slower execution time. In the end, clean code is not necessarily fast code.

That said, this test does not really hit on the strength of Scala, which is scaling multi-threaded environments. With less than 8 hours experience writing Scala, I'm sure my code is far from efficient.

Update: I realized that the default behavior of the parsers is to validate the Schema. I've disabled this behavior in the .Net version but not the Scala/Java version. Much of the 3 seconds to 'load' the XML is really an external HTTP request and the validation. However, using the Scala library I do not see an easy way to disable the Schema validation.

Apple's plist XML (Properties List) is PAINFUL

I started poking around with Scala and tried to re-implement the iTunes Music Library parser I wrote in .Net as a Scala library to give me a real problem to solve. This effort reminded me of how painful Apple's plists are to work with.

Apple saves the iTunes Library information in a binary file and as well as an XML file using its plist format. It is great that the provide the XML data that enables tools like iTunesExport, but they certainly could have made working with the XML easier.

Simply put, they store key value pairs as:
<key>Artist</key><string>U2</string>
Three is no clean way to associate the 'value' with a 'key' other than the order of the nodes in XML. This makes things like XPath parsing very difficult.

For a more in depth explanation, read this post.

I was able to get the Scala library to parse the plist but it wasn't as clean as I hoped. If I stay motivated I may finish up a Scala version of the tool to provide a version that will work on OS X (Scala compiles down to Java classes).

Google Provide Personal Transportation Device

Google posted an interesting 6 minute video tour of their new container data center. It's a reasonably interesting video, but my favorite part is the "Google provided personal transportation device".

The inside views of the containers are interesting. Obviously all the equipment here is tailored for maximum efficiency.

HDR Photography

I decided to try creating some HDR photographs during my recent trip (with my new Tripod).

Here are a few samples (using Photomatix). Click the images for full size views.

Original 'normal exposure' image:


HDR Image (Tone Compressor):


HDR Image (Detail Enhancer):


There are tons of tweaks with both Tone Compressor and Detail Enhancer so you can make each one look radically different. For these landscape shots both HDR options look much richer than the original normal exposure. The Detail Enhancer is more surreal and obviously not a traditional photograph where I think the Tone Compressor done right could look like a traditional picture (just with more dynamic range than normally seen).

Tripod: Gitzo 2932

I recently bought a Gitzo 2930 Tripod. It is part of their Basalt line, which is between Aluminum and Carbon Fiber. I couldn't justify the cost difference for a Carbon Fiber tripod but I wanted a light tripod that would be with me for years.

So far, I've been very pleased with the results. I carried it down into the Grand Canyon on my recent hike for some self portraits and bracketed shots for HDR.

I really like the build quality and locking mechanism for the legs (the kind you twist, but they work and require only about 1/2 turn). I can setup the tripod in seconds and it has been stable for everything I need.

This model does come with a center column to extend up, which I recently used to get a picture from an angle I could not have otherwise achieved. I had the camera up above my head angled down, and used live view to compose the shot. The hook to attach a weight/bag really came in handy when I did this. I was really surprised at how much additional stability it provided.

Achilles vs. the Grand Canyon

No, I didn't hike to the bottom.

But, after my Achilles surgery this past November I was very excited to get a real hike in. I finished my physical therapy/rehab a few weeks ago and I am cleared for most activities that don't involve jumping.

We started at the South Rim and hiked down 3 miles on the Bright Angel Trail. 3 Miles may not seem like a lot (6 round trip), but it was also a 2,100 ft. drop (and climb) in elevation. My ankle was a bit sore (as were a few other muscles) but overall my ankle did fine and I'm happy to be active again.

Here is a shot back up the canyon from the 3 mile rest house.

Amazon Prime

Amazon Prime is awesome. Amazon Prime is simply this: you pay $79 for a year and they will ship all of your purchases 2nd Day for FREE. And if you can have your order overnight for $3.99 per item.

If you make enough purchases it pays off, and with the service they've provided recently it is certainly worthwhile.

I ordered my Kindle on Sunday evening. When I got home Tuesday it was waiting at my doorstep. I didn't overnight it, this was the 'normal' 2 day shipping.

I ordered Joe McNally's latest flash book Hot Shoe Diaries on Wednesday, and toyed with overnight delivery so it would be here before I went on vacation. But the estimated shipping date wasn't until next week so I just went with the normal 2 day shipping. I got the email that it shipped today and was slightly disappointed that I didn't overnight it. To my SHOCK, it arrived Friday evening by MESSENGER SERVICE. Amazon sent the book to my by messanger for free.

I'm not sure how they're making money on this, but I don't see a better way to buy.

Kindle 2 Second Impression

I posted my first impression with the Kindle a few weeks ago and I wanted to follow up with my thoughts now that I've had some real time to use it.

I love it. I've read more in the last few weeks that I did in the previous 6 months, and while the newness of the Kindle is part of it, I believe the convenience will stand the test of time.

My thoughts here will be about the Kindle, but will be more focused on how it changes my behavior. At the end of the day* a new gadget isn't about how many pixels it has, but how using it changes what you do.

I now take the Kindle with me anytime I might be be waiting 5 minutes or more. I'm in Physical Therapy from my Achilles surgery, and I started reading during the warm up and ice down parts (10 minutes each). These are small periods of time, but escaping into another world makes the time much more enjoyable, and makes me a happier person.

I carry it in my work bag every day, just in case. I would never do this with a real book.

I find it much easier to read than a laptop, or even a book. Once you get the hang of it, it is really easy to curl up with the Kindle and comfortably page through your book with very little effort. I really noticed the difference when I took a break from my Kindle and checked my RSS reader on my laptop. It just wasn't as comfortable or enjoyable of an experience, and I found it much harder to read even moderately long posts.

I've read books through the Amazon store ('Daemon', and 'Predictably Irrational') and books available freely (an Algorithms book I've had on my laptop for years and have not read, and Cory Doctorow's 'Down and Out in the Magic Kingdom' and 'Eastern Standard Tribe' (just started this today).).

It is easy to convert books into Amazon compatible formats using either Amazon's free service (or pay if you have it delivered directly) or Mobipocket Creator (Non-DRM Mobi files are compatible with the Kindle).

It is easy to read. The page turning delay/flashing does not bother me (it may help that I have the older Sony Reader as a baseline) and the text/background is very readable in good light. My daughter was sick today and I spent the better part of the day reading the Kindle with no more eyestrain that I would expect reading a normal book.

It is easier to hold than a real book.

The battery is good enough. I'm not sure it'll last a week with real use but it certainly lasts most of a week. I did start turning off the Wireless (very easy to turn off and on) when I'm not using it to save battery life. I started today with a full battery and am still 3/4 after reading nearly the entire day (without wireless most of the day).

The built in dictionary is great. I've used it at least once with every book I've read (yes, my vocabulary is that bad). The web browser is passable, mostly useful to display Wikipedia pages when the dictionary is insufficient.

I have not tested any newspaper or magazine subscriptions yet. I currently read the Economist (on paper) but I plan on suspending my subscription and testing out the two week trial subscription to the Wall Street Journal (the Economist is not available through the Kindle store). We'll see how this turns out, but I look forward to not having to deal with propping the Economist open so I can read hands free while I eat my breakfast. We'll see...

All in all, I'm very happy with the Kindle 2. I'm headed on a relaxing week long vacation soon and am very excited to carry just the Kindle instead of a bag full of books. It is impossible to run out of books to read with the built in Amazon Store.


* I put this in there for you Brad.

Predictably Irrational

I am about 1/2 of the way through Predictably Irrational and am really enjoying it. While I think much of the topics covered fall into the obvious/previously covered category, it is interested to be reminded at how predictably irrational we truly are.

The first chapter 'The Truth about Relativity' covers concepts I've known (I've been known to (over)use the term 'it's all relative'), but provides empirical evidence that really demonstrates how predictable irrational behavior is. It really puts 'keeping up with the Jones'' in perspective.

The second chapter covers the concept of an anchor price. Our perception of what what something is worth is based on our first purchase or near purchase of that item, again with some interesting experiments to support the ideas.

Chapter Three covers the cost of zero, or why we alter our behavior when something is FREE FREE FREE!.

Chapter Four is another chapter that rings true for me. The title is 'The Cost of Social Norms', IE when it is a bad idea to offer to pay your Mom for Thanksgiving dinner! But for me it reminded me of my experiences developing Open Source software, and how they changed when I got paid.

I could have read the book straight through, but I wanted to take a break so some of the ideas could sink in before continuing. It is amazing how much reading you can get done when you have a sick kid sleeping on your lap all day!

Kindle 2 First Impressions

I broke down and bought a Kindle.

My interested in eBooks started when a friend loaned me his Sony Reader. It was cool, but not compelling enough. The first version of the Kindle was a breakthrough, but I held off for version 2. Well, version 2 is here.

My first impression when I saw the box tucked in my front door was that that can't possibly be the Kindle, the box was too small! The packaging is great and certainly in the vein of Apple's minimalistic packaging. The box contains three items, the Kindle, a combined charger/USB cable, and a quick start guide. The full users guide is an eBook on the Kindle (of course).

I was immediately impressed with the increased speed of the Kindle 2 vs. the Sony Reader. The Sony's page refreshes were so slow the menus were nearly unusable. The Kindle feels very snappy by comparison. I'm not sure how it compares to the original Kindle other than the quoted '20%' faster. When reading, the Kindle 2 changes pages as fast as you could turn a page in a real book.

The built in dictionary function is very easy to use and unobtrusive. It is a great idea and shows how eBooks can be better, not just 'as good as' real books. The ability to search Wikipedia in addition to the current book, the whole Kindle, and Google is great. The browser is passable.

My biggest issue is that I have to go finish the paper book I'm currently reading before I can really dig into a book on the Kindle! I'll post a more complete review once I've used it for a while.

And thanks to Jeff's Kindle posts for pushing me over the edge, even though he's sticking with his OLD Kindle. :)

Google cleans up URLs

I'm a little 'uptight' about URLs. I've blogged about my frustration with the Java Servlet mapping constraints before, and I'm a believer in W3C recommendations on URLs.

Google just announced a tool for Webmasters that allows users to specify a canonical URL for a single 'page', even if you can access it from multiple URLs. This is a nice patch for systems that don't really conform to the above W3C recommendation but believe in the concept. It also helps clean up Google's indexes, which is another good thing.

Since their approach is a simple tag added the the section of an HTML document, hopefully the other search engines will adopt it as well. The example from their post:
<link rel="canonical" href="http://www.example.com/product.php?item=swedish-fish" />

Appcelerator

I came across Appcelerator this week (thanks Andy).
Appcelerator is an open source platform that provides everything you need to build rich web, mobile and desktop applications.
From what I saw Appcelerator is a thin layer on the Javascript API making it much easier to program. It also comes with a set of pre-defined effects that can provide some quick pop to your applications.

A great feature is the 'Appcelerator in 5 Minutes' demo. If you need 5 minutes with it to 'get' it, you're a little slow. You can pretty much learn the model in 60 seconds there. The rest is seeing the scope of functionality.

What I found even more interesting is the approach. Appcelerator views the browser as its deployment environment, and interacts with many different backends, including Java, .Net, Ruby, PHP, and more. This separation is an interesting approach and actually matches in concept what James Strachan was talking about in his post 'JAX-RS as the one Java web framework to rule them all?'. Developing web applications consists of two parts, the UI (HTML, JavaScript, CSS, Images), and the backend (Java, .Net, PHP, Ruby, accessed via JSON, REST, XML, etc). Why do we need to build them as one, or with a single technology?

As with anything, there are applications that this approach works well for (Gmail -> GWT takes an all in one approach, but does treat the browser as a 'real' deployment environment), and many it does not. If nothing else, Appcelerator is on my list to spend some more time with.