Wednesday, February 29, 2012

Clojure in pure Python is a great idea

Edit: Here's the link

Not good enough for me to do it myself, but I'd be very happy if someone else did it. Why?

You'd give up Java interoperability, but you'd gain interoperability with native code. Python offers excellent C++ interoperability via Boost and other methods, as well as excellent (but limited by the constraints of C) interoperability with Fortran. Further, rpy embeds, more or less, R in Python. It would make Clojure much more suitable for numerical work. For that matter, you could call all of scipy from Clojure. Awesome.


Another reason is that it would reduce the amount of competition between languages. I hate language wars because they're a waste of time. I might prefer one language for a given task, but I use many, so using one doesn't mean you can't use another. Anything that allows you to write in both Python and Clojure is helpful - let the developer make the choice, even on individual sections of code.

Another big advantage is getting us away from Java. The build tools, the classpath, the .com.lengthy.verbose.wtf.on.and.on are inherited from Java, and seriously suck. I guess tail call optimization is possible.

So you could say I'm a fan. Provided that it is possible to implement nearly all of Clojure, to the point that you can be confident that you will be able to write Clojure code for the JVM and expect it to run on the Python VM. It would be necessary to add functions such as Math/pow to get to that point. A lot of work, but definitely valuable. I hope it is viewed as an additional option for Clojure developers, not a threat to the JVM.

{Why am I writing this here? I put all comments on my blog. I don't have the time or interest to get into an internet shouting match.}

11 comments:

Anonymous said...

Interesting idea but I think it would be a backwards step - one of the reasons that Clojure is so awesome is that the JVM is an extremely powerful platform - IMHO easily the best platform in the world for general purpose server-side computing.

You can also call native code from the JVM as well so that's not an argument for moving off the JVM.

I can certainly see the potential appeal of mixing Clojure and Python, but I'd suggest doing it on the JVM with Clojure + Jython (which already both exist) rather than trying to reinvent the wheel anywhere else....

lmf said...

We'll have to disagree on this one. The problem with your comment is that Java is a drawback rather than an advantage for many tasks - in particular numerical computing. It's far from the best platform. Clojure in its current form is basically a platform for writing web apps. That's fine, but not all apps are web apps.

You can call native code from the JVM. Getting everything to work properly is about as much fun as hitting yourself in the side of your head with a hammer. Plus there's always going to be a performance hit, which is relevant if you're repeatedly calling a C function inside a loop.

Anonymous said...

Why not Clojure on Objective-C? Much faster than python with presumably even better interaction with C libraries. Harder to do, yes, but not impossible.

Also, suggesting that Java and Clojure by extension is only good for writing web apps is a ridiculously extreme statement. There are plenty of scientific software libraries written in Java and the Java C interface is decent enough for most applications outside of the use case you describe (calling a C function repeatedly in a tight loop or tricky cross-memory access).

Clojure on python is neat, but it sounds like what you really want is Clojure in a C variant.

lmf said...

Clojure on python is neat, but it sounds like what you really want is Clojure in a C variant.

That's correct, but I'd go further and say Python is the best piece of glue we've got. It's very easy to work with code in C, C++, Fortran, and R, plus you get scipy for free.

suggesting that Java and Clojure by extension is only good for writing web apps is a ridiculously extreme statement

It is quite clear that Rich Hickey and Clojure/Core are focused like a laser on web apps. Of course you can do other things, but for me those other things are numerical, and there's just not much activity in that area in the Clojure community.

I'll put out a challenge to you or anyone else: show me an example where you easily create Java bindings to the GSL. I've wasted enough time to know that it's not fun at all.

Then give me a list of Java and Clojure replacements for everything in scipy.

Pretending that Clojure can interoperate with native code can only slow Clojure adoption. There are not a lot of programmers out there who are going to put in that kind of effort.

lmf said...

Just to be clear, the difficulty with calling C/C++/Fortran code is mostly due to C/C++/Fortran and not Java/Clojure. Preprocessor macros are just brutal. Swig doesn't handle nested classes and overloaded operators. I could go on, but it all sucks, especially if you can use another language where the bindings already exist.

I would love to be wrong, but unfortunately I think I'm right.

Anonymous said...

Don't get me wrong, I'm not claiming the C bindings are great for what you are doing with the gsl, numpy, and others. But I've had very little problem using jvm clojure for NLP work binding with C conditional random field libraries or using some of the good Java NLP libraries.

There are a very wide range of problems for which Java works just fine outside of the web and your area. For all of these clojure is still well suited.

That said, I myself would love to see a C version of clojure. The only downside would be no standard set of existing libraries that can be imported with no work. Bindings would have to be made for the different C libraries. This may seem like a weak argument, but I think good easy to use library supports is one of the downsides to common lisp and others. There are some efforts to fix it, but it doesn't seem to be gaining much traction overall.

Python would bring that plus decent C bindings, but going from the JVM to the python VM is really unsatisfying. I'd like native clojure code to be faster, not slower.

In any case, it doesn't hurt to have a python version of clojure so long as some care is taken not to fragment the language too much.

Anonymous said...

What about Python's GIL restrictions?

I like Python, and used it to implement a store, forward, and reporting system for water meter reads. This also included a simple Django web site.

However, one of the reasons I started learning Clojure was because of its concurrent programming features.

lmf said...

You're right, that could be a problem. I would be interested to hear how they plan to address the issue.

Cory Giles said...

Would be way too much effort to reimplement everything. See for example the languishing status of the .NET implementation.

A more realistic way forward is to improve the Clojure toolchain so that it's more amenable to easily create apps that can be used on the shell alongside your code that uses C/Python+GSL.

There are many impediments to this: as one example, the way leiningen is currently designed, you can't pipe stdout to a Clojure process (unless of course you uberjar first). What is needed, imo, are more projects in the vein of cljr, jark, or even lein oneoff, that can manage Clojure libs/classpaths in a more global and portable way than lein.

Anonymous said...

We stumbled over here coming from a different website and thought I might
as well check things out. I like what I
see so now i'm following you. Look forward to looking over your web page for a second time.

My web site: free online backup service
Feel free to surf my webpage best online backup service

Anonymous said...

This is the perfect blog for anyone who wishes to find out about this topic.
You realize a whole lot its almost hard to argue with you (not that I actually
would want to…HaHa). You definitely put a brand new spin on a topic that has
been discussed for decades. Wonderful stuff, just great!


My blog post; online computer backup
My homepage :: free online backup