Is Scala Too Complex?

Is Scala too complex to become the 'New Java'?  This question has been debated before, and there are some good existing posts including:

• Is Scala really more complicated than Java? - Dick Wall (with discussion)
• Is Scala more complicated than Java? - Vassil Dichev

I tend to mostly agree with their arguments that Scala is not inherently more complex, but instead is different and has complexity in different ways.  The syntax is different, and you need to learn a few different rules, and some features like Traits (multi-inheritance) and implicit conversions can make tracking down how things work a bit more difficult.  In the end though, I don't believe Scala is itself more complex than Java.

But I want to take a different angle on this question.  I believe that the use of Scala to build Domain Specific Languages illustrates how some of Scala's features can be used to create code that can be complex to understand, but that may be OK.

Domain Specific Languages
Domain Specific Languages, or DSLs, provide a syntax suited to a specific problem domain.  Scala provides a set of features that enable the development of Scala libraries that enable a DSL or pseudo-DSL.

Lift is a web framework written in Scala.  It provides what I consider a Scala based DSL for creating web applications.  Let's examine some sample code from a simple Lift Snippet.  (A Lift Snippet is somewhat related to a Servlet):

class A {
  def snippet (xhtml : NodeSeq) : NodeSeq = bind("A", xhtml, "name" -> Text("The A snippet"))
}

This example (from Exploring Lift site - pdf) provides a trivial snippet implementation.  It simply replaces the <A:name /> XML tag with "The A snippet" text and would be called by the following XHTML:

<lift:A.snippet>
  <p>Hello, <A:name />!</p>
</lift:A.snippet> 

There are a few things going on here to make this work. First, the bind method is statically imported. The import looks like:

import net.liftweb.util.Helpers._

This imports all the methods defined in the Helpers class into your class, enabling bind to be called without prefix. The Helpers class itself is just a roll-up of 9 other Helper class, so you could instead:

import net.liftweb.util.BindHelpers._

or even:

import net.liftweb.util.BindHelpers.bind

if you want strict control over what you are including.  However, the point here is not that you can do static imports, which are possible in Java as well, but that the Lift framework makes heavy use of static imports to help create its DSL.

The second part requires two important concepts, implicit functions and operator overloading.  The bind method has a few overloaded versions, but a typical case uses the following signature:

def bind(namespace : String, xml : NodeSeq, params : BindParam*)

In our simple example, the first two parameters are pretty straight forward, but how does:

"name" -> Text("The A snippet")

Get converted to a BuildParam?  The answer is implicit functions and operator overloading.  The BindHelpers Object (statically imported with Helpers._), contains an implicit method with the following signature:

implicit def strToSuperArrowAssoc(in : String) : SuperArrowAssoc

The SuperArrowAssoc object defines a method (operator overload) ->.  This operator/method is overloaded to take different parameters, including scala.xml.Text. So in our simple example:

"name" -> Text("The A snippet")

The "name" string is implicitly converted to a SuperArrowAssoc, and the -> method is executed with a scala.xml.Text instance as the parameter, and returns a BindParam, which is then passed to the statically imported bind method.

Got all that?

It is a bit complicated, and it took me a little while to track down how it all worked.  However...  There are two important points to make before concluding that this is 'bad'.

First, the disparity between IDE functionality for Scala and Java is enormous.  It is reasonably easy to understand any piece of code quickly in Java, in large part because of the IDE.  It is trivial to understand inheritance trees, all references to a method/variable,  all subclasses, etc.  What I would like to see in Scala is easy decoding/indication of implicit methods and static imports.  The Scala 2.8 plugin for Eclipse still fails at handling basic Class and import statement auto-complete most of the time, let alone handling more complex values⁽¹⁾.

Second, and I think more interestingly, I'm not sure it matters if you 'understand' the code.  The point of a DSL is to make it easy to build something in a specific domain.  Yes, you are coding Scala, but do you really need to know where/how everything is defined/wired?  If I'm building a web site, do I care how "x" -> "y" gets converted and handled by Lift?  What you have is a case where you end up learning the DSL syntax.  I think this is different than learning a library API in Java.  A library API uses traditional Java syntax, so you are just learning what methods exit.  In Lift, you have to learn syntax itself, because there is no obvious way to 'discover' what is possible with the Lift library.  An example:

Using Java and a traditional Library, I can use auto-complete to find the 'primary' classes, and use method auto-complete to look at the methods.  Assuming it is a reasonably designed API, I can probably figure out how to use it without much more.  Of course reference thing JavaDoc will help and is reccomended to make sure everything works as you assume, but you can get a long way on your own.

In Lift, there is no easy way to auto-discover how to interact with the framework using the DSL.  You must read documentation/samples to understand what is available, or dig through endless pages of ScalaDoc to deduce the possible options.

In the end, I don't believe Scala is inherently more complex than Java.  That said, I think it is easier to write hard to understand Scala code than it is with Java.  Writing a DSL in Scala can make for code that is nearly impossible to 'infer proper usage' from the traditional means, but that isn't surprising.  It is a DOMAIN SPECIFIC LANGUAGE, in essence, you created a new syntax, and you have to acknowledge that that learning that is different than learning the language it is written in.

I remain a fan of Lift and Scala.  Even with these issues, I feel that the value they deliver is worth the learning curve necessary to become proficient. 

(1) - But I am still very appreciative that the plugin exists at all, and hope they keep up the hard work!