April 23, 2009
In an earlier post I compared a .Net and Scala implementation of an iTunes Music Library XML parser. The Scala version took a long time to ’load the XML file from disk’. Since then I realized that the default behavior of the parsers is to validate the Schema. I had disabled this behavior in the .Net version but not the Scala/Java version. Much of the 3 seconds to ’load’ the XML is really an external HTTP request to download the xsd file and perform the validation. However, using the Scala library I do not see an easy way to disable the Schema validation (Under the covers the Scala library is delegating to existing Java APIs).
April 21, 2009
Oracle and Sun announced an agreement for Oracle to acquire Sun. What do I think?
It’s better than IBM. And it is aimed at IBM.
IBM and Sun seemed to have a lot of overlap. Certainly IBM has a well defined hardware business, and acquiring Sun’s hardware business doesn’t really help. IBM is heavily invested in Java so I’m sure they would have enjoyed more influence over its growth and direction, but it didn’t really seem compelling.
Oracle on the other hand puts the final piece of the puzzle together. Oracle is now a credible Enterprise IT partner. With the addition of Sun’s hardware business, Oracle can compete with IBM at nearly every level. If I’m a large company, I can turn to Oracle for:
Storage Solutions
Servers
Database
Enterprise Software Packages
Custom Enterprise Software Development (Java, WebLogic, etc.)
If you are a large company, you can either hand your checkbook over to Oracle or IBM. Or Microsoft+HP.
As a user of Java and MySQL, I’m hopeful that they will at least continue on as is and hopefully improve. I don’t expect any real changes in either for a while though.
April 15, 2009
As part of my iTunesExport utility I wrote a (C#.Net) module to parse the iTunes XML file and provide two simple collections, Playlists and Tracks. As part of my effort to learn Scala, I rewrote this module in Scala. While the libraries don’t provide the exact same interface and functinality, they are effectively the same. The stats:
(Physical) Line of Code count:
.Net: 459
Scala: 226
The Scala version accomplished the same functionality in 1/2 the lines of code. While LOC count is a bit arbitrary, I think it is an important point. The Scala code is much more concise, but maintains or even improves the readability (if you have a reasonable familiarity with Scala). In this specific case Scala’s handling of public properties provided a big reduction. Ex:
Scala:
val trackId: Double = …
.Net:
private string id;
public string Id
{
get{ return id; }
}
While it may seem that this provides much of the savings, the Scala version provides a much more verbose toString method, so some of the per-property savings is actually understated in this comparison.
While developing I did find myself consistently following the pattern:
- Identify logical functionality I wanted to extract into a helper method
- Writing code for helper method
- Realize it was just a single line and moving back inline
Simply put, I found the structure and features of the Scala language very well suited to write high level understandable code that performed the same functionality in fewer lines of code, compared to C# or Java.
Performance was something else. As a functional language Scala is well suited to be a scalable language. It also runs on the JVM, benefiting from a significant amount of work building a high performance virtual environment. That said, this implemenation was purely single threaded. The initial results were somewhat surprising:
Scala 4,800 ms
.Net 650 ms
That seemed like a pretty significant difference, so I took a deeper look. The Scala library to load the XML file from disk took nearly 3 seconds! This is a huge hit and accounts for much of the performance difference. The remining functionality took ~1,800 ms, nearly 3x the entire .Net solution. Where does the time go?
~3000 ms - reading the XML file from disk using Scala’s XML library
~100 ms - parsing the main library attributes.
~1700 ms - parsing the tracks
~60 ms - parsing the playlists
Clearly loading the file and parsing the tracks are the biggest targets. I’ll leave loading the XML out of scope as it is a Scala library and focus on the tracks. For each of entities (Library, Playlist, Track) I use a Trait that parses the plist XML into a Map and then assign the properties from the map. For each track, parsing the XML into a Map took 1 ms or less, but assigning the variables from the map (and doing any type conversions) took up to 4 ms but more commonly took 1 ms or less. At the single ms level the simple act of measuring introduces a significant impact, so I’m not sure these numbers are meaningful.
With over 2600 tracks in the library, even spending 1 ms per track is forever (and indeed the average time per track without the measurement was closer to .65 ms). The additional complexity introduced by my Scala approach caused a significantly slower execution time. In the end, clean code is not necessarily fast code.
That said, this test does not really hit on the strength of Scala, which is scaling multi-threaded environments. With less than 8 hours experience writing Scala, I’m sure my code is far from efficient.
Update: I realized that the default behavior of the parsers is to validate the Schema. I’ve disabled this behavior in the .Net version but not the Scala/Java version. Much of the 3 seconds to ’load’ the XML is really an external HTTP request and the validation. However, using the Scala library I do not see an easy way to disable the Schema validation.
April 14, 2009
I started poking around with Scala and tried to re-implement the iTunes Music Library parser I wrote in .Net as a Scala library to give me a real problem to solve. This effort reminded me of how painful Apple’s plists are to work with.
Apple saves the iTunes Library information in a binary file and as well as an XML file using its plist format. It is great that the provide the XML data that enables tools like iTunesExport, but they certainly could have made working with the XML easier.
Simply put, they store key value pairs as:
<key>Artist</key><string>U2</string>
Three is no clean way to associate the ‘value’ with a ‘key’ other than the order of the nodes in XML. This makes things like XPath parsing very difficult.
For a more in depth explanation, read this post.
I was able to get the Scala library to parse the plist but it wasn’t as clean as I hoped. If I stay motivated I may finish up a Scala version of the tool to provide a version that will work on OS X (Scala compiles down to Java classes).
April 10, 2009
Google posted an interesting 6 minute video tour of their new container data center. It’s a reasonably interesting video, but my favorite part is the “Google provided personal transportation device”.
The inside views of the containers are interesting. Obviously all the equipment here is tailored for maximum efficiency.
March 26, 2009
I decided to try creating some HDR photographs during my recent trip (with my new Tripod).
Here are a few samples (using Photomatix). Click the images for full size views.
Original ’normal exposure’ image:
HDR Image (Tone Compressor):
HDR Image (Detail Enhancer):
There are tons of tweaks with both Tone Compressor and Detail Enhancer so you can make each one look radically different. For these landscape shots both HDR options look much richer than the original normal exposure. The Detail Enhancer is more surreal and obviously not a traditional photograph where I think the Tone Compressor done right could look like a traditional picture (just with more dynamic range than normally seen).
March 26, 2009
I recently bought a Gitzo 2930 Tripod. It is part of their Basalt line, which is between Aluminum and Carbon Fiber. I couldn’t justify the cost difference for a Carbon Fiber tripod but I wanted a light tripod that would be with me for years.
So far, I’ve been very pleased with the results. I carried it down into the Grand Canyon on my recent hike for some self portraits and bracketed shots for HDR.
I really like the build quality and locking mechanism for the legs (the kind you twist, but they work and require only about 1/2 turn). I can setup the tripod in seconds and it has been stable for everything I need.
This model does come with a center column to extend up, which I recently used to get a picture from an angle I could not have otherwise achieved. I had the camera up above my head angled down, and used live view to compose the shot. The hook to attach a weight/bag really came in handy when I did this. I was really surprised at how much additional stability it provided.
March 25, 2009
No, I didn’t hike to the bottom.
But, after my Achilles surgery this past November I was very excited to get a real hike in. I finished my physical therapy/rehab a few weeks ago and I am cleared for most activities that don’t involve jumping.
We started at the South Rim and hiked down 3 miles on the Bright Angel Trail. 3 Miles may not seem like a lot (6 round trip), but it was also a 2,100 ft. drop (and climb) in elevation. My ankle was a bit sore (as were a few other muscles) but overall my ankle did fine and I’m happy to be active again.
Here is a shot back up the canyon from the 3 mile rest house.
March 21, 2009
Amazon Prime is awesome. Amazon Prime is simply this: you pay $79 for a year and they will ship all of your purchases 2nd Day for FREE. And if you can have your order overnight for $3.99 per item.
If you make enough purchases it pays off, and with the service they’ve provided recently it is certainly worthwhile.
I ordered my Kindle on Sunday evening. When I got home Tuesday it was waiting at my doorstep. I didn’t overnight it, this was the ’normal’ 2 day shipping.
I ordered Joe McNally’s latest flash book Hot Shoe Diaries on Wednesday, and toyed with overnight delivery so it would be here before I went on vacation. But the estimated shipping date wasn’t until next week so I just went with the normal 2 day shipping. I got the email that it shipped today and was slightly disappointed that I didn’t overnight it. To my SHOCK, it arrived Friday evening by MESSENGER SERVICE. Amazon sent the book to my by messanger for free.
I’m not sure how they’re making money on this, but I don’t see a better way to buy.
March 9, 2009
I posted my first impression with the Kindle a few weeks ago and I wanted to follow up with my thoughts now that I’ve had some real time to use it.
I love it. I’ve read more in the last few weeks that I did in the previous 6 months, and while the newness of the Kindle is part of it, I believe the convenience will stand the test of time.
My thoughts here will be about the Kindle, but will be more focused on how it changes my behavior. At the end of the day* a new gadget isn’t about how many pixels it has, but how using it changes what you do.
I now take the Kindle with me anytime I might be be waiting 5 minutes or more. I’m in Physical Therapy from my Achilles surgery, and I started reading during the warm up and ice down parts (10 minutes each). These are small periods of time, but escaping into another world makes the time much more enjoyable, and makes me a happier person.
I carry it in my work bag every day, just in case. I would never do this with a real book.
I find it much easier to read than a laptop, or even a book. Once you get the hang of it, it is really easy to curl up with the Kindle and comfortably page through your book with very little effort. I really noticed the difference when I took a break from my Kindle and checked my RSS reader on my laptop. It just wasn’t as comfortable or enjoyable of an experience, and I found it much harder to read even moderately long posts.
I’ve read books through the Amazon store (’Daemon’, and ‘Predictably Irrational’) and books available freely (an Algorithms book I’ve had on my laptop for years and have not read, and Cory Doctorow’s ‘Down and Out in the Magic Kingdom’ and ‘Eastern Standard Tribe’ (just started this today).).
It is easy to convert books into Amazon compatible formats using either Amazon’s free service (or pay if you have it delivered directly) or Mobipocket Creator (Non-DRM Mobi files are compatible with the Kindle).
It is easy to read. The page turning delay/flashing does not bother me (it may help that I have the older Sony Reader as a baseline) and the text/background is very readable in good light. My daughter was sick today and I spent the better part of the day reading the Kindle with no more eyestrain that I would expect reading a normal book.
It is easier to hold than a real book.
The battery is good enough. I’m not sure it’ll last a week with real use but it certainly lasts most of a week. I did start turning off the Wireless (very easy to turn off and on) when I’m not using it to save battery life. I started today with a full battery and am still 3/4 after reading nearly the entire day (without wireless most of the day).
The built in dictionary is great. I’ve used it at least once with every book I’ve read (yes, my vocabulary is that bad). The web browser is passable, mostly useful to display Wikipedia pages when the dictionary is insufficient.
I have not tested any newspaper or magazine subscriptions yet. I currently read the Economist (on paper) but I plan on suspending my subscription and testing out the two week trial subscription to the Wall Street Journal (the Economist is not available through the Kindle store). We’ll see how this turns out, but I look forward to not having to deal with propping the Economist open so I can read hands free while I eat my breakfast. We’ll see…
All in all, I’m very happy with the Kindle 2. I’m headed on a relaxing week long vacation soon and am very excited to carry just the Kindle instead of a bag full of books. It is impossible to run out of books to read with the built in Amazon Store.
* I put this in there for you Brad.
