Clojure/Lisp Readability

Tim Bray has put up an excellent set of theses on Clojure. I can agree with most of them, but wonder about the idea that "Lisp is a handicap".

I understand where Tim comes from, and I fully admit that even after playing with Lisp, Scheme and Clojure for quite some time, I'm still not sure whether this is really an issue. I would only feel qualified to really comment on it once I have used it on a real project with some complexity. But what I found interesting is the example Tim gives:

(apply merge-with +
  (pmap count-lines
    (partition-all *batch-size*
      (line-seq (reader filename)))))

I find this particular example to be extremely readable if you read it from the inside out - the reader function presumably returns a reader for the file named filename; line-seq returns a (lazy) sequence of the lines in the file, partition-all cuts this into segments (using *batch-size* as the, well, batch size), pmap maps the function count-lines over the result in parallel, returning a list of maps; finally, merge-with + combines all of the values for the same keys in the map by adding them. (Obviously the source code is much more understandable than my prose version.)

I agree there is a ton of Lisp code out there that's intimidating, but I don't see this as a good example. Let's invent a language with similar expressiveness, but more traditional syntax:

apply(mergeWith(plus),
  pmap(countLines, 
    partitionAll(BATCH_SIZE,
     lineSeq(reader(filename)))));

I can't see how this would be easier to read. It's not the syntax (at least not in this case) that might be a problem if you're not used to it, it's the style of using nested function invocations – which I believe is something you get used to really quickly if using any kind of language that supports functional idioms.

Comments

On December 2, 2009 11:04 AM, symphonious.net

said:

I’d say the real problem with both of those versions is that they implement pretty much the entire wide-finder project in one line of code. Breaking it up makes it much more readable (and puts it mostly back in the right order).

var lines = lineSeq(reader(filename)); var processedData = pmap(countLines, partitionAll(BATCH_SIZE, lines)); var result = mergeWith(plus, processedData);

You can vary how much or how little you break it down to get the balance right, but this is dramatically more readable than the original because you can see the three key components - get the data and split it into lines, process the data in parallel and finally put it all back together. Extracting some methods for those three lines to give them more readable names may be well worth while too.

Cheers,

Adrian Sutton.

On December 2, 2009 11:38 AM, http://www.innoq.com/blog/st/

said:

It’s a matter of taste to a certain degree, but in any case even your version would look very similar if it were written in Clojure.

On December 2, 2009 12:20 PM, Daniel Sobral said:

It’s a language thing. English speakers (and those of many other languages) are used to see subject first, then verb, and Lisp inverts that. That same snippet written in Scala (minus pmap), would read like this:

Source fromPath “scotch.scala” getLines “\n” grouped 5 map (.size) reduceLeft (+_)

For better or worse, most people find this more natural.

On December 2, 2009 1:05 PM, Rich Hickey said:

Yes, as you say, it can be a style or familiarity issue. As you demonstrated, you can write nested calls in any language, and people do, up to a certain degree of complexity. Then they will (and should) switch away from ‘inside out’. The same is true in Clojure, which supports multiple sequential styles cleanly.

The same code in a pipelined Clojure style:

(->> (line-seq (reader filename))
  (partition-all *batch-size*)
  (pmap count-lines)
  (apply merge-with +))

and in a step-wise Clojure style with labeled interim results (a la Adrian’s comment):

(let [lines (line-seq (reader filename))
      processed-data (pmap count-lines 
                           (partition-all *batch-size* lines))]
  (apply merge-with + processed-data))

Most Clojure programmers will be familiar with and mix these three styles, preferring, I hope, the one that makes the code clearest in a given situation.

Regards,

Rich

On December 3, 2009 12:19 AM, http://www.innoq.com/blog/st/

said:

@Rich: Honored to have you pay a visit here, and thanks for the pipelining example.

On December 5, 2009 1:07 PM, dasuxullebt said:

Yet another discussion about the readability of Lisp. When will we ever learn to ignore this type of criticism, and accept that there are people who like lisp-syntax, and people who dont?