I am not afraid to admit that I've used Lisp for real work

[Really long post, but I don't see how to break it into smaller pieces. I wrote it a few weeks ago. I didn't want to post it at that time because I wrote some negative things about certain Lisp implementations. I took out a few of the negative comments, but most of them are still there. I stand by my opinions. You're free to have your own opinions, just like me, but I don't publish comments with swear words. I also don't publish comments that are uncivil or completely uninformative.]

As is the case with most programmers, I'd heard of Lisp, but didn't take it seriously because it was always presented as a dead language. Many of those comments came from former Lisp users. Lisp was just not up to the standards of "modern" languages, like Java, C++, Python, Ruby, (add your favorite language). Lots of nostalgia, but they'd made peace with the death of Lisp the way you make peace with the death of a pet dog when you're in middle school.

As is also the case for many programmers, I came upon the writings of Paul Graham and decided that Lisp might be worthy of further exploration. Then I read that the only app anyone admits having written in Lisp was later rewritten in Python. (That's only a slight exaggeration.)

It's hard to come upon examples of the use of Lisp the way you can find examples of Ruby, Python, or even Tcl.

Let me proudly say that I'm not afraid to write programs in Lisp. My situation is different from, say, a web developer for a Fortune 500 corporation. I do numerical programming. I get paid to do simulations, statistical analysis, and a variety of other types of numerical analysis.

I have one of the best jobs in the world. I am free to choose my programming language. It makes no difference if I'm the only programmer in the world using a particular language. Most of the programs I write are less than a few thousand lines, and most of the time, they are almost independent of all the other programs I write. In other words, what matters is that the program I write for a specific task does its job for that task and that task alone. Yep, I do scripting.

Which languages do I use? If you have read other posts on this blog (making you part of a very small group) you know R is what I use most often. Yet I love Lisp and plan to do a lot more Lisp programming going forward. I would not be surprised if five years from now Lisp was my main programming language.

Lisp is a great language and you too can trust it if you write numerical programs of 5000 lines or less. You can probably trust it for other tasks as well but I don't have such an experience with Lisp. Please tell the world that Lisp is a great language for numerical/statistical analysis. Those who dismiss it are wrong. There's absolutely no reason to not give it a trial. See what libraries are available. Look at some code written by others. It never hurts to learn another language, and if you have used only Matlab, you will learn a hell of a lot about programming even if the only thing you do is read The Scheme Programming Language without writing a single line of Lisp code. You have a lot to gain and nothing to lose. I guarantee it.*

Now that I've given a little background, I'd like to discuss the question of which Lisp to use. There are a number of alternatives in the Lisp world. The choice can be confusing to beginners, which would have included me not that long ago. I will briefly discuss four of the available options: Common Lisp, Scheme, Clojure, and newLISP.

Common Lisp: This one is often described as the "industrial strength" Lisp. In many cases, someone will just say "Lisp" when in fact they are referring to Common Lisp. My assessment of Common Lisp is that it's what you'd have if you start with Java, take out most of the good stuff, add on some Lisp features in the clumsiest possible way, and leave changes to a political process that ensures change will never happen.

Common Lisp is sort of the Oldsmobile of programming languages. It was good relative to the alternatives in the mid-80's, but then it stagnated, leading to its downfall.

There are some advantages. It's the only Lisp that's ever been popular in the enterprise. There's a lot of documentation available. There are a lot of libraries available. Some of the compilers have the potential to produce really fast code. There are, possibly surprisingly to the outsider, a lot of programmers who have used Common Lisp. That helps to the extent that you need to interact with other Lisp users, say to ask questions.

The community is a serious drawback. While most of the community is great, they haven't figured out how to control a vocal few.

My opinion: Stay away from Common Lisp as a beginner. The user community is stuck in the disco era. You can always try it later if you want. If you try only Common Lisp you will agree with those who say Lisp is not a suitable alternative to today's more popular languages. It had its chance to be popular, it failed spectacularly, and it failed spectacularly for a reason. If you have a beard and 15 years of Unix experience, you will probably feel right at home, so my advice doesn't apply to you.

Scheme: My first use of Scheme to any degree was going through SICP. Absolutely awesome experience.

I've written elsewhere about the big problem with Scheme. It's elegant, but the core language is too small to do anything useful, and the extensions are of exceptionally low quality, with some of the worst documentation of any product in the western hemisphere.

Scheme is an excellent language for teaching programming. The Scheme developers do not have an incentive to put together high-quality, properly documented language extensions. This is what you will encounter if you read the documentation on a Scheme extension:

foo [option] name code

foo transforms name into code. You can set options as desired.

Section 5 of the Scheme specification prohibits the use of informative examples when documenting functions. The 80th time I had to look at source code to figure out what an extension did was when I gave up on Scheme.

The developers of Scheme extensions are happy when they can say, "It is possible to do X" for any choice of X. To be useful, they'd need to be able to say, "It is possible to do X in a reasonable amount of time with a reasonable amount of effort." You sure as heck don't want to be relying on a piece of code and then find out that it's not doing what you thought it was, and have no idea how to fix the problem. That's when you start thinking about the reaction of your family if you get fired.

And no, Racket is not different in that regard, the extensions are a complete mess and the examples, if they exist at all, are unlikely to help much. This is not a criticism of Scheme or Racket. They are intended to be used as tools for teaching and research. I'm fine with that. There are other choices available. If you are learning Lisp do not get frustrated that Scheme/Racket can't do what you need. That does not mean Lisp is a bad choice of language.

Clojure: This is one awesome language. It's a modern Lisp. Not only is the language awesome, but the creator, Rich Hickey, has a lot of interesting things to say. You can tell that he has been in the trenches for a long time doing real programming, yet he is excited to incorporate ideas from the academic community.

Oh, Clojure also runs on the JVM, meaning you have access to everything provided by the king of enterprise programming languages. There are several good Clojure books out already, and more are on the way. There is serious upside both in terms of the language and the prospects for a career writing Lisp programs.

There are at least two downsides. The first is that using Clojure almost guarantees you will be bringing in some Java. That means many of the bad things about Java apply to Clojure as well. With time, Clojure is likely to develop its own libraries, which will minimize the Java drawbacks. The second downside is that the development tools are not mature, as you should expect with such a young language.

newLISP: Ultimately, I have come to like newLISP the best. newLISP is unmatched in the Lisp world in terms of documentation. After dealing with Common Lisp and Scheme documentation, I was shocked at the quality and quantity of examples that come with newLISP. You can learn to do everything the language does with little difficulty.

Another attractive feature of newLISP is that Common Lisp programmers criticize it for failing to meet their definition of being a "real" Lisp. That's a good sign, because Common Lisp has failed, so if you want to put together a language programmers actually want to use you better do some things differently from Common Lisp. newLISP lets you get work done quickly and correctly, and that's the only reason I've ever used a programing language.

As a scripting language, newLISP is not a competitor to either Common Lisp or Clojure. It is not a speed demon, but if you're using a scripting language, you obviously should not expect the speed of C or Java. It does quite well in terms of speed relative to other scripting languages I've used including Tcl and Python.

For most Linux distributions you will have to compile it yourself. However, that is easy (almost trivial - just follow the directions) and extremely fast (measured in seconds rather than minutes).

The bottom line: If you want to get started with Lisp for real world projects and have programming experience, install newLISP and try it out. The documentation on the website will definitely help you to get going quickly.

If you want to learn to program, install Racket, and use drracket with SICP. Any Scheme will also do.

If you want something suitable for the enterprise, get one of the Clojure books and give that a try.

Although I don't have any recommend uses for Common Lisp, there are many resources available if you want to go that route. CLISP is a good choice, but keep in mind that it does not come with a compiler. I will add that Land of Lisp is a very, very good book (and recent to boot) that will be one of the best purchases you ever make.

The decision of which Lisp is not as big as it might seem. You can carry a lot of your knowledge from one to the other. The best strategy may well be to study all of them. Whichever you choose you will be happy with your choice - even if that choice turns out to be Common Lisp.

*I'm an anonymous blogger, so don't expect my guarantee to mean much.