[Really long post, but I don't see how to break it into smaller pieces. I wrote it a few weeks ago. I didn't want to post it at that time because I wrote some negative things about certain Lisp implementations. I took out a few of the negative comments, but most of them are still there. I stand by my opinions. You're free to have your own opinions, just like me, but I don't publish comments with swear words. I also don't publish comments that are uncivil or completely uninformative.]
As is the case with most programmers, I'd heard of Lisp, but didn't take it seriously because it was always presented as a dead language. Many of those comments came from former Lisp users. Lisp was just not up to the standards of "modern" languages, like Java, C++, Python, Ruby, (add your favorite language). Lots of nostalgia, but they'd made peace with the death of Lisp the way you make peace with the death of a pet dog when you're in middle school.
As is also the case for many programmers, I came upon the writings of Paul Graham and decided that Lisp might be worthy of further exploration. Then I read that
the only app anyone admits having written in Lisp was later rewritten in
Python. (That's only a slight exaggeration.)
It's hard to come upon examples of the use of Lisp the way you can find examples of Ruby, Python, or even Tcl.
Let me proudly say that I'm not afraid to write programs in Lisp. My situation is different from, say, a web developer for a Fortune 500 corporation. I do numerical programming. I get paid to do simulations, statistical analysis, and a variety of other types of numerical analysis.
I have one of the best jobs in the world. I am free to choose my programming language. It makes no difference if I'm the only programmer in the world using a particular language. Most of the programs I write are less than a few thousand lines, and most of the time, they are almost independent of all the other programs I write. In other words, what matters is that the program I write for a specific task does its job for that task and that task alone. Yep, I do scripting.
Which languages do I use? If you have read other posts on this blog (making you part of a very small group) you know R is what I use most often. Yet I love Lisp and plan to do a lot more Lisp programming going forward. I would not be surprised if five years from now Lisp was my main programming language.
Lisp is a great language and you too can trust it if you write numerical programs of 5000 lines or less. You can probably trust it for other tasks as well but I don't have such an experience with Lisp. Please tell the world that Lisp is a great language for numerical/statistical analysis. Those who dismiss it are wrong. There's absolutely no reason to not give it a trial. See what libraries are available. Look at some code written by others. It never hurts to learn another language, and if you have used only Matlab, you will learn a hell of a lot about programming even if the only thing you do is read The Scheme Programming Language without writing a single line of Lisp code. You have a lot to gain and nothing to lose. I guarantee it.*
Now that I've given a little background, I'd like to discuss the question of which Lisp to use. There are a number of alternatives in the Lisp world. The choice can
be confusing to beginners, which would have included me not that long
ago. I will briefly discuss four of the available options: Common Lisp, Scheme, Clojure, and newLISP.
Common Lisp: This one is often described as the "industrial strength" Lisp. In many cases, someone will just say "Lisp" when in fact they are referring to Common Lisp. My assessment of Common Lisp is that it's what you'd have if you start with Java, take out most of the good stuff, add on some Lisp features in the clumsiest possible way, and leave changes to a political process that ensures change will never happen.
Common Lisp is sort of the Oldsmobile of programming languages. It was good relative to the alternatives in the mid-80's, but then it stagnated, leading to its downfall.
There are some advantages. It's the only Lisp that's ever been popular in the enterprise. There's a lot of documentation available. There are a lot of libraries available. Some of the compilers have the potential to produce really fast code. There are, possibly surprisingly to the outsider, a lot of programmers who have used Common Lisp. That helps to the extent that you need to interact with other Lisp users, say to ask questions.
The community is a serious drawback. While most of the community is great, they haven't figured out how to control a vocal few.
My opinion: Stay away from Common Lisp as a beginner. The user community is stuck in the disco era. You can always try it later if you want. If you try only Common Lisp you will agree with those who say Lisp is not a suitable alternative to today's more popular languages. It had its chance to be popular, it failed spectacularly, and it failed spectacularly for a reason. If you have a beard and 15 years of Unix experience, you will probably feel right at home, so my advice doesn't apply to you.
Scheme: My first use of Scheme to any degree was going through SICP. Absolutely awesome experience.
I've written elsewhere about the big problem with Scheme. It's elegant, but the core language is too small to do anything useful, and the extensions are of exceptionally low quality, with some of the worst documentation of any product in the western hemisphere.
Scheme is an excellent language for teaching programming. The Scheme developers do not have an incentive to put together high-quality, properly documented language extensions. This is what you will encounter if you read the documentation on a Scheme extension:
foo [option] name code
foo transforms name into code. You can set options as desired.
Section 5 of the Scheme specification prohibits the use of informative examples when documenting functions. The 80th time I had to look at source code to figure out what an extension did was when I gave up on Scheme.
The developers of Scheme extensions are happy when they can say, "It is possible to do X" for any choice of X. To be useful, they'd need to be able to say, "It is possible to do X in a reasonable amount of time with a reasonable amount of effort." You sure as heck don't want to be relying on a piece of code and then find out that it's not doing what you thought it was, and have no idea how to fix the problem. That's when you start thinking about the reaction of your family if you get fired.
And no, Racket is not different in that regard, the extensions are a complete mess and the examples, if they exist at all, are unlikely to help much. This is not a criticism of Scheme or Racket. They are intended to be used as tools for teaching and research. I'm fine with that. There are other choices available. If you are learning Lisp do not get frustrated that Scheme/Racket can't do what you need. That does not mean Lisp is a bad choice of language.
Clojure: This is one awesome language. It's a modern Lisp. Not only is the language awesome, but the creator, Rich Hickey, has a lot of interesting things to say. You can tell that he has been in the trenches for a long time doing real programming, yet he is excited to incorporate ideas from the academic community.
Oh, Clojure also runs on the JVM, meaning you have access to everything provided by the king of enterprise programming languages. There are several good Clojure books out already, and more are on the way. There is serious upside both in terms of the language and the prospects for a career writing Lisp programs.
There are at least two downsides. The first is that using Clojure almost guarantees you will be bringing in some Java. That means many of the bad things about Java apply to Clojure as well. With time, Clojure is likely to develop its own libraries, which will minimize the Java drawbacks. The second downside is that the development tools are not mature, as you should expect with such a young language.
newLISP: Ultimately, I have come to like newLISP the best. newLISP is unmatched in the Lisp world in terms of documentation. After dealing with Common Lisp and Scheme documentation, I was shocked at the quality and quantity of examples that come with newLISP. You can learn to do everything the language does with little difficulty.
Another attractive feature of newLISP is that Common Lisp programmers criticize it for failing to meet their
definition of being a "real" Lisp. That's a good sign, because Common Lisp
has failed, so if you want to put together a language programmers
actually want to use you better do some things differently from Common
Lisp. newLISP lets you get work done quickly and correctly, and that's the only reason I've ever used a programing language.
As a scripting language, newLISP is not a competitor to either Common Lisp or Clojure. It is not a speed demon, but if you're using a scripting language, you obviously should not expect the speed of C or Java. It does quite well in terms of speed relative to other scripting languages I've used including Tcl and Python.
For most Linux distributions you will have to compile it yourself. However, that is easy (almost trivial - just follow the directions) and extremely fast (measured in seconds rather than minutes).
The bottom line: If you want to get started with Lisp for real world projects and have programming experience, install newLISP and try it out. The documentation on the website will definitely help you to get going quickly.
If you want to learn to program, install Racket, and use drracket with SICP. Any Scheme will also do.
If you want something suitable for the enterprise, get one of the Clojure books and give that a try.
Although I don't have any recommend uses for Common Lisp, there are many resources available if you want to go that route. CLISP is a good choice, but keep in mind that it does not come with a compiler. I will add that Land of Lisp is a very, very good book (and recent to boot) that will be one of the best purchases you ever make.
The decision of which Lisp is not as big as it might seem. You can carry a lot of your knowledge from one to the other. The best strategy may well be to study all of them. Whichever you choose you will be happy with your choice - even if that choice turns out to be Common Lisp.
*I'm an anonymous blogger, so don't expect my guarantee to mean much.
Friday, August 12, 2011
Monday, July 11, 2011
Why not use Scheme more?
Another observation post, probably not worth your time, but I want to express my thoughts. You've been warned.
I really first started using Scheme when I went through SICP. Well, I didn't make it all the way through, but I learned a lot in any event. When you've got masters like that to show you what can be done with Scheme/Lisp you wonder why everyone isn't using it.
As you can guess, I think I know why not everyone is using it. I primarily do numerical computing, so that is the set of features I consider when evaluating a new programming language. A couple of things stand out when I think about using Scheme rather than R and Fortran.
1. There is a lack of libraries. This is well known, sometimes called "the island problem".
2. There is a lack of data structures. If I want to do matrix algebra, I want to use a language that offers a matrix class as part of the language. I want to use a language that does matrix multiplication as part of the language. There are good reasons for that: less hassle, less chance of breakage when the core of the language changes, less chance that you're relying on an extension written by one individual that will stop working when he gets bored and moves on.
Item 1 is not as true as it used to be. The major Schemes have a lot of libraries available. There are also tools for wrapping C code, so honestly, any lack of libraries more or less reflects the small user base, rather than the other way around. If there were more interest, the libraries would appear quickly. There's just no interest.
Item 2 is still a problem and likely to remain so forever. I'm just not sure that a minor inconvenience is a major hurdle to adoption. Something like a standard set of libraries being available with each implementation would be a solution.
I've reconsidered the situation since discovering Clojure, and IMO the real problem with Scheme (any of them) is the sucky documentation. It stinks. It really, really stinks. {Clarification: Upon reading the post, I was not clear that I'm talking about the developers of Scheme extensions, not the developers of the core language itself. There exists very good documentation for the core language if you're doing the usual things.}
Have the Scheme documentation writers never heard of examples?
I've never seen a community that was so collectively clueless about how to write documentation. And I'm a Slackware user. I don't think anyone has ever complained about the excessive documentation that comes with Slackware.
I might even consider writing some documentation myself, if not for the fact that I can never get anything in Scheme to work, because the documentation sucks so bad. My philosophy is to just move on if I have to work too hard to figure something out. I've come to realize that developers that don't document their work very well are usually not good developers.
Let me elaborate on that point a bit. The software might work as intended, but the underlying code is only part of the project, and by itself has no value. A good design means I don't have to think too much to use it. And when I return to the code in a year, I'll be able to easily figure out what I did and how to extend it.
If I have to work for six hours to figure out how to call a simple function, that's a sign that at some point I'm going to find that my program isn't doing what I thought it was doing. At some point I'm going to say, "WTF? Why was that matrix transposed? It makes no sense." Of course there was no way I could have known that, and there's no way I will ever understand the philosophy driving the development of the library. Then a few seconds later I'll be terrified, "Hold on, if the developer would do something like that, how can I trust the other lines of code that I've been writing for the last two weeks? There's no freaking documentation!" Then I'll move on to a real language, redo everything, and be sure to avoid the mistake of trusting Scheme in the future.
Go to one of the popular Scheme implementations. Just pick one. They're all equally useless. Pick out an extension that does something you need for numerical computing. If there's documentation at all - and there may very well not be - try to use that documentation to write a program. Use all of the important "things" that the extension does. Use those important things in different ways, like you would actually use them in an actual programming exercise where you are actually trying to feed yourself. Can you even figure out how to call all of the functions properly in less than two weeks?
Most of the documentation is just a slopped-together list of functions and a short, content-free description that might (though in most cases probably won't) be sufficient to jog the memory of the developer himself as to what the function should do. There's no chance in hell that a programmer writing a program related to his career is going to spend enough time to figure out how to use the extension. Sure, there are hobbyists who will figure it out, but I'm talking about real work in an environment where money is on the line.
I cannot treat the developers of Scheme as real developers. They're not professionals. Some are, of course, but I'm talking averages here, and the average Scheme developer is of pretty low quality relative to the average Java developer. The average Scheme developer thinks he's much better than the average Java developer because he can use recursion properly and was smart enough to recognize the brilliance of Lisp. Great, but if you measure developer quality in terms of the value of the output in a given amount of programmer time, Scheme developers probably have a negative value, because most of what they do gets in the way of developers interested in making a product that does something of value.
Do I like Java? Not a chance in hell. If I had to choose between Scheme and Java to do something with money on the line, even assuming all the same libraries are available in Scheme and Java, which would I choose? I'd take Java in a second. Scheme could learn a lot from Java about working with beginners.
It's a shame that Scheme has been plagued by crappy documentation. There's no reason that Scheme, on technical merit, cannot be Java. The difference is that Java was written for business applications, and that required documentation, and they got it. The Scheme community has decided that working with beginners is not fun and it has caused Scheme to be viewed as a goofy artifact of the past, rather than the language of the future.
I really first started using Scheme when I went through SICP. Well, I didn't make it all the way through, but I learned a lot in any event. When you've got masters like that to show you what can be done with Scheme/Lisp you wonder why everyone isn't using it.
As you can guess, I think I know why not everyone is using it. I primarily do numerical computing, so that is the set of features I consider when evaluating a new programming language. A couple of things stand out when I think about using Scheme rather than R and Fortran.
1. There is a lack of libraries. This is well known, sometimes called "the island problem".
2. There is a lack of data structures. If I want to do matrix algebra, I want to use a language that offers a matrix class as part of the language. I want to use a language that does matrix multiplication as part of the language. There are good reasons for that: less hassle, less chance of breakage when the core of the language changes, less chance that you're relying on an extension written by one individual that will stop working when he gets bored and moves on.
Item 1 is not as true as it used to be. The major Schemes have a lot of libraries available. There are also tools for wrapping C code, so honestly, any lack of libraries more or less reflects the small user base, rather than the other way around. If there were more interest, the libraries would appear quickly. There's just no interest.
Item 2 is still a problem and likely to remain so forever. I'm just not sure that a minor inconvenience is a major hurdle to adoption. Something like a standard set of libraries being available with each implementation would be a solution.
I've reconsidered the situation since discovering Clojure, and IMO the real problem with Scheme (any of them) is the sucky documentation. It stinks. It really, really stinks. {Clarification: Upon reading the post, I was not clear that I'm talking about the developers of Scheme extensions, not the developers of the core language itself. There exists very good documentation for the core language if you're doing the usual things.}
Have the Scheme documentation writers never heard of examples?
I've never seen a community that was so collectively clueless about how to write documentation. And I'm a Slackware user. I don't think anyone has ever complained about the excessive documentation that comes with Slackware.
I might even consider writing some documentation myself, if not for the fact that I can never get anything in Scheme to work, because the documentation sucks so bad. My philosophy is to just move on if I have to work too hard to figure something out. I've come to realize that developers that don't document their work very well are usually not good developers.
Let me elaborate on that point a bit. The software might work as intended, but the underlying code is only part of the project, and by itself has no value. A good design means I don't have to think too much to use it. And when I return to the code in a year, I'll be able to easily figure out what I did and how to extend it.
If I have to work for six hours to figure out how to call a simple function, that's a sign that at some point I'm going to find that my program isn't doing what I thought it was doing. At some point I'm going to say, "WTF? Why was that matrix transposed? It makes no sense." Of course there was no way I could have known that, and there's no way I will ever understand the philosophy driving the development of the library. Then a few seconds later I'll be terrified, "Hold on, if the developer would do something like that, how can I trust the other lines of code that I've been writing for the last two weeks? There's no freaking documentation!" Then I'll move on to a real language, redo everything, and be sure to avoid the mistake of trusting Scheme in the future.
Go to one of the popular Scheme implementations. Just pick one. They're all equally useless. Pick out an extension that does something you need for numerical computing. If there's documentation at all - and there may very well not be - try to use that documentation to write a program. Use all of the important "things" that the extension does. Use those important things in different ways, like you would actually use them in an actual programming exercise where you are actually trying to feed yourself. Can you even figure out how to call all of the functions properly in less than two weeks?
Most of the documentation is just a slopped-together list of functions and a short, content-free description that might (though in most cases probably won't) be sufficient to jog the memory of the developer himself as to what the function should do. There's no chance in hell that a programmer writing a program related to his career is going to spend enough time to figure out how to use the extension. Sure, there are hobbyists who will figure it out, but I'm talking about real work in an environment where money is on the line.
I cannot treat the developers of Scheme as real developers. They're not professionals. Some are, of course, but I'm talking averages here, and the average Scheme developer is of pretty low quality relative to the average Java developer. The average Scheme developer thinks he's much better than the average Java developer because he can use recursion properly and was smart enough to recognize the brilliance of Lisp. Great, but if you measure developer quality in terms of the value of the output in a given amount of programmer time, Scheme developers probably have a negative value, because most of what they do gets in the way of developers interested in making a product that does something of value.
Do I like Java? Not a chance in hell. If I had to choose between Scheme and Java to do something with money on the line, even assuming all the same libraries are available in Scheme and Java, which would I choose? I'd take Java in a second. Scheme could learn a lot from Java about working with beginners.
It's a shame that Scheme has been plagued by crappy documentation. There's no reason that Scheme, on technical merit, cannot be Java. The difference is that Java was written for business applications, and that required documentation, and they got it. The Scheme community has decided that working with beginners is not fun and it has caused Scheme to be viewed as a goofy artifact of the past, rather than the language of the future.
Saturday, July 09, 2011
What's wrong with Clojure syntax?
Short post, not a lot of value.
I keep reading things about Clojure's syntax being ugly. I don't think that's the case at all. It's extremely consistent and easy to follow. Maybe I'm just weird, but I've always preferred Tcl for short scripts because I like the syntax. Sometimes the critics of Clojure/Lisp syntax are even Java programmers!
I keep reading things about Clojure's syntax being ugly. I don't think that's the case at all. It's extremely consistent and easy to follow. Maybe I'm just weird, but I've always preferred Tcl for short scripts because I like the syntax. Sometimes the critics of Clojure/Lisp syntax are even Java programmers!
Saturday, June 18, 2011
Calling R from Clojure
From what I've read and based on the time I've played around with it, Clojure looks like a very good language. There's no reason to praise it here. Just ask Google if you want to read some praise for Clojure.
For me, it is promising because it runs on the JVM and has complete access to existing Java code. That means it's both fast and will do most of what you want to do. (Scala fits into the same category. I will be checking out Scala in the future.)
In addition, it is a dialect of Lisp, which means I'm a huge fan of the syntax as wells as the approach to problem solving. I've enjoyed using Scheme in the past, but given the lack of libraries, Scheme is not suitable for the numerical programming I do. I'm aware of projects like Chicken Scheme and Gambit-C, and I know about Swig and their foreign function interfaces and all that, but I'm too busy to do everything my employer wants me to do. There's no way I'm going to spend months of my time extending the language, usually with crappy documentation if it even exists at all.
I said above that Java will do "most" of what you want to do because one of the few limitations of Java is that it is just not complete in terms of numerics. The situation is better than it was ten years ago, but I've still not worked much with Java, simply because I know it won't do everything I need it to do. Oh, and I hate the syntax with a passion. 8000 lines of GOTO-infested FORTRAN 77 is much more pleasant to me than 200 lines of Java.
The Clojure community offers a project called incanter that provides access to many of the available Java math libraries. There's also a project called rincanter that is supposed to bring R functionality to Clojure, building on the Java Rserve client. If it worked I would be happy to use it. Unfortunately rincanter does not work right now.
An Example
It's easy enough to get it working in Clojure. Here's what I did, along with a short example code.
First, I installed Eclipse. I then installed counterclockwise (no idea why they chose that name). I went through the tutorial instructions to set up a working project. All of my previous Clojure programs were REPL, so I just went through the experience of setting up the Eclipse environment, and can happily say that it is easy to do. I then downloaded Rserve.jar and Rengine.jar from rforge.
After starting my new project, I told Eclipse about Rserve.jar and Rengine.jar by right-clicking on "Referenced Libraries" --> "Build Path" --> "Configure Build Path..." --> "Add External JARs...".
Here's a sample program to demonstrate the calling syntax. You can convert the other parts of the Rserve Java examples into Clojure using the same ideas.
(import '(org.rosuda.REngine))
(import '(org.rosuda.REngine.Rserve RConnection))
(def c (new RConnection))
(def d (. c eval "rnorm(10)"))
(def e (. d asDoubles))
(println (first e))
(println (rest e))
This was done using Clojure 1.2 and Eclipse 3.5.2 on Linux Mint 11.
For me, it is promising because it runs on the JVM and has complete access to existing Java code. That means it's both fast and will do most of what you want to do. (Scala fits into the same category. I will be checking out Scala in the future.)
In addition, it is a dialect of Lisp, which means I'm a huge fan of the syntax as wells as the approach to problem solving. I've enjoyed using Scheme in the past, but given the lack of libraries, Scheme is not suitable for the numerical programming I do. I'm aware of projects like Chicken Scheme and Gambit-C, and I know about Swig and their foreign function interfaces and all that, but I'm too busy to do everything my employer wants me to do. There's no way I'm going to spend months of my time extending the language, usually with crappy documentation if it even exists at all.
I said above that Java will do "most" of what you want to do because one of the few limitations of Java is that it is just not complete in terms of numerics. The situation is better than it was ten years ago, but I've still not worked much with Java, simply because I know it won't do everything I need it to do. Oh, and I hate the syntax with a passion. 8000 lines of GOTO-infested FORTRAN 77 is much more pleasant to me than 200 lines of Java.
The Clojure community offers a project called incanter that provides access to many of the available Java math libraries. There's also a project called rincanter that is supposed to bring R functionality to Clojure, building on the Java Rserve client. If it worked I would be happy to use it. Unfortunately rincanter does not work right now.
An Example
It's easy enough to get it working in Clojure. Here's what I did, along with a short example code.
First, I installed Eclipse. I then installed counterclockwise (no idea why they chose that name). I went through the tutorial instructions to set up a working project. All of my previous Clojure programs were REPL, so I just went through the experience of setting up the Eclipse environment, and can happily say that it is easy to do. I then downloaded Rserve.jar and Rengine.jar from rforge.
After starting my new project, I told Eclipse about Rserve.jar and Rengine.jar by right-clicking on "Referenced Libraries" --> "Build Path" --> "Configure Build Path..." --> "Add External JARs...".
Here's a sample program to demonstrate the calling syntax. You can convert the other parts of the Rserve Java examples into Clojure using the same ideas.
(import '(org.rosuda.REngine))
(import '(org.rosuda.REngine.Rserve RConnection))
(def c (new RConnection))
(def d (. c eval "rnorm(10)"))
(def e (. d asDoubles))
(println (first e))
(println (rest e))
This was done using Clojure 1.2 and Eclipse 3.5.2 on Linux Mint 11.
Tuesday, December 21, 2010
Starting R to run Rmpi jobs
I always forget the proper command to start R to run Rmpi jobs. Here it is:
mpirun -np 1 --hostfile [name of hosts file] R --no-save
The thing that always throws me is that I use -np 2 or whatever number of processors. I forget that you need to start one instance of R and then from within that one instance you can access however many processors you have in your hosts file.
Note: You get the same thing by using "orterun" rather than "mpirun" in the above command, but I like mpirun because it is obvious what it is doing.
mpirun -np 1 --hostfile [name of hosts file] R --no-save
The thing that always throws me is that I use -np 2 or whatever number of processors. I forget that you need to start one instance of R and then from within that one instance you can access however many processors you have in your hosts file.
Note: You get the same thing by using "orterun" rather than "mpirun" in the above command, but I like mpirun because it is obvious what it is doing.
Monday, October 04, 2010
Slow PDF printing in Linux
I've struggled with slow printing of PDF files in Linux recently. It's only been in recent months though. I do not recall having this problem before.
I finally realized that it was only a problem to print PDF files that are scanned images. The solution was buried in this thread:
http://ubuntuforums.org/archive/index.php/t-1073942.html
I set up a second printer. It is parallel, so (in Gnome) I went to System > Administration > Add
I selected "Other". I needed a Device URI. A google search suggested for parallel printers that parallel:/dev/lp0 might work. I tried it, used "Generic" for the manufacturer, and chose PCL 5e as the driver.
Works great. All PDF images print fast if I remember to choose the Generic printer rather than the HP.
I finally realized that it was only a problem to print PDF files that are scanned images. The solution was buried in this thread:
http://ubuntuforums.org/archive/index.php/t-1073942.html
I set up a second printer. It is parallel, so (in Gnome) I went to System > Administration > Add
I selected "Other". I needed a Device URI. A google search suggested for parallel printers that parallel:/dev/lp0 might work. I tried it, used "Generic" for the manufacturer, and chose PCL 5e as the driver.
Works great. All PDF images print fast if I remember to choose the Generic printer rather than the HP.
Tuesday, August 03, 2010
Cuda on Slackware 13.1
This was much easier than I expected. Note that I'm running Slackware 13.1 32-bit.
Step 1: Install the latest NVidia driver (256.44 in my case). You can't use the SlackBuild from slackbuilds.org. The latest NVidia packages are quite different from 195.*. I uninstalled the existing driver (that I had earlier installed from slackbuilds.org), downloaded the latest NVidia driver, and installed it according to the instructions.
Step 2: Install the CUDA toolkit for Fedora 12. This amounted to downloading the file and running it. I accepted all the defaults.
Step 3: Install the CUDA SDK. Again, I downloaded the file, ran it, and accepted the defaults.
Step 4: Change the paths according to the CUDA instructions.
Done! This was so much easier than Ubuntu, Mint, Debian, and others. I didn't expect it to work by just downloading a few files and running them. I'm certainly not complaining. I ran make for the examples, they all compiled and ran perfectly.
Step 1: Install the latest NVidia driver (256.44 in my case). You can't use the SlackBuild from slackbuilds.org. The latest NVidia packages are quite different from 195.*. I uninstalled the existing driver (that I had earlier installed from slackbuilds.org), downloaded the latest NVidia driver, and installed it according to the instructions.
Step 2: Install the CUDA toolkit for Fedora 12. This amounted to downloading the file and running it. I accepted all the defaults.
Step 3: Install the CUDA SDK. Again, I downloaded the file, ran it, and accepted the defaults.
Step 4: Change the paths according to the CUDA instructions.
Done! This was so much easier than Ubuntu, Mint, Debian, and others. I didn't expect it to work by just downloading a few files and running them. I'm certainly not complaining. I ran make for the examples, they all compiled and ran perfectly.
Wednesday, July 28, 2010
Gretl on Slackware 13.1
I recently built gretl 1.9.1 for Slackware 13.1. I needed to install two packages first, fftw and gdk-pixbuf. Both are available from slackbuilds.org.
LAPACK and BLAS are also necessary. I prefer to use the ACML because it is a lot faster than an unoptimized BLAS. I created symbolic links to make libacml.a appear to be both libblas.a and liblapack.a in /usr/lib:
ln -s /opt/acml4.4.0/gfortran32/lib/libacml.a /usr/local/lib/libblas.a
ln -s /opt/acml4.4.0/gfortran32/lib/libacml.a /usr/local/lib/liblapack.a
Here's the SlackBuild. It built a package and upon installation it runs correctly.
LAPACK and BLAS are also necessary. I prefer to use the ACML because it is a lot faster than an unoptimized BLAS. I created symbolic links to make libacml.a appear to be both libblas.a and liblapack.a in /usr/lib:
ln -s /opt/acml4.4.0/gfortran32/lib/libacml.a /usr/local/lib/libblas.a
ln -s /opt/acml4.4.0/gfortran32/lib/libacml.a /usr/local/lib/liblapack.a
Here's the SlackBuild. It built a package and upon installation it runs correctly.
#!/bin/shLicense note: This is a modified version of a SlackBuild downloaded from slacky.eu a long time ago. It is in the public domain, as I understand it, and my modifications are public domain as well.
# Heavily based on the Slackware 12.2 SlackBuild
# http://gretl.sourceforge.net/gretl_italiano.html
NAME=gretl
VERSION=1.9.1
ARCH=${ARCH:-i486}
BUILD=1mf
CWD=`pwd`
if ["$TMP" = ""]; then
TMP=/tmp
fi
PKG=$TMP/package-$NAME
if [ "$ARCH" = "i486" ]; then
SLKCFLAGS="-O2 -march=i486 -mtune=i686"
elif [ "$ARCH" = "i686" ]; then
SLKCFLAGS="-O2"
elif [ "$ARCH" = "x86_64" ]; then
SLKCFLAGS="-O2"
fi
if [ ! -d $TMP ]; then
mkdir -p $TMP
fi
if [ ! -d $PKG ]; then
mkdir -p $PKG
fi
mkdir -p $PKG/usr
cd $TMP
tar xvjf $CWD/$NAME-$VERSION.tar.bz2
cd $NAME-$VERSION
CFLAGS="$SLKCFLAGS" \
./configure --prefix=/usr --build=i486-slackware-linux --with-lapack-prefix=/usr
make
make prefix=$PKG/usr install
mkdir -p $PKG/usr/doc/$NAME-$VERSION
cp -a ChangeLog COPYING EXTENDING INSTALL README README.audio README.win32 TODO $PKG/usr/doc/$NAME-$VERSION
mkdir -p $PKG/install
mkdir -p $PKG/usr/share/pixmaps
mkdir -p $PKG/usr/share/applications
cat $CWD/slack-desc > $PKG/install/slack-desc
cp $CWD/gretl.png $PKG/usr/share/pixmaps/gretl.png
cat $CWD/Gretl.desktop > $PKG/usr/share/applications/gretl.desktop
cp $CWD/$NAME.SlackBuild $PKG/usr/doc/$NAME-$VERSION/
( cd $PKG
find . | xargs file | grep "executable" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
find . | xargs file | grep "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
)
mv $PKG/usr/share/man $PKG/usr
gzip -9 $PKG/usr/man/*/*
chmod 755 $PKG/usr/bin/*
cd $PKG
chown -R root:root .
requiredbuilder -v -y -s $CWD $PKG
cat $CWD/slack-required > $PKG/install/slack-required
cat $CWD/slack-required > $PKG/usr/doc/$NAME-$VERSION/slack-required
makepkg -l y -c n $CWD/$NAME-$VERSION-$ARCH-$BUILD.tgz
if [ "$1" = "--cleanup" ]; then
rm -rf $TMP/$NAME-$VERSION
rm -rf $PKG
fi
Monday, January 18, 2010
Simple clamav tutorial
To install clamav on Slackware 13, I used the SlackBuilds.org SlackBuild.
To update the database:
freshclam
To scan all files in directory dir:
cd /dir
clamscan -r -i
This will scan all files in that directory and all subdirectories, and will only print out the names of infected files.
To update the database:
freshclam
To scan all files in directory dir:
cd /dir
clamscan -r -i
This will scan all files in that directory and all subdirectories, and will only print out the names of infected files.
Restart networking in Slackware 13
Couldn't easily find it, so here it is. As root, run
/etc/rc.d/rc.inet1 eth0_restart
You may have to change the "eth0" if your interface goes by a different name.
/etc/rc.d/rc.inet1 eth0_restart
You may have to change the "eth0" if your interface goes by a different name.
Monday, September 28, 2009
gretl on Slackware 13.0
I had some difficulties getting gretl 1.8.4 installed on Slackware 13. Here is what I did.
The problem was with the LAPACK libraries. After a lot of digging, I found that the problem was twofold: (a) I had built my own LAPACK library using the SlackBuild from slackbuilds.org, and gretl wasn't looking for the library in the right directory. (b) It wasn't finding libblas.a. (You can open config.log for gretl in the /tmp/gretl-1.8.4 directory to find the configuration problems.)
To handle the first problem, I had to edit the SlackBuild script to add the configure option --with-lapack-prefix=/usr to tell it to look for liblapack.a in /usr/lib. It will look in the /lib subdirectory of the directory you give, so if you pass the option --with-lapack-prefix=/usr/lib, it will look in /usr/lib/lib and quit with an error.
The second problem was caused by the fact that I had built my LAPACK library against ACML, AMD's optimized math libraries. I created a symbolic link of that library into /usr/local/lib. (There are probably alternatives, but I know it will be found if I put it there.) I had installed ACML in the default directory, so the symbolic link I created was
ln -s /opt/acml4.3.0/gfortran32/lib/libacml.a /usr/local/lib/libblas.a
The gretl SlackBuild built a package that installed properly and ran, but gave an error when I attempted to plot a graph. I don't know what the error was but it was fixed when I ran gretl from a Slackware 12.1 installation. After that, gretl plotting worked.
The problem was with the LAPACK libraries. After a lot of digging, I found that the problem was twofold: (a) I had built my own LAPACK library using the SlackBuild from slackbuilds.org, and gretl wasn't looking for the library in the right directory. (b) It wasn't finding libblas.a. (You can open config.log for gretl in the /tmp/gretl-1.8.4 directory to find the configuration problems.)
To handle the first problem, I had to edit the SlackBuild script to add the configure option --with-lapack-prefix=/usr to tell it to look for liblapack.a in /usr/lib. It will look in the /lib subdirectory of the directory you give, so if you pass the option --with-lapack-prefix=/usr/lib, it will look in /usr/lib/lib and quit with an error.
The second problem was caused by the fact that I had built my LAPACK library against ACML, AMD's optimized math libraries. I created a symbolic link of that library into /usr/local/lib. (There are probably alternatives, but I know it will be found if I put it there.) I had installed ACML in the default directory, so the symbolic link I created was
ln -s /opt/acml4.3.0/gfortran32/lib/libacml.a /usr/local/lib/libblas.a
The gretl SlackBuild built a package that installed properly and ran, but gave an error when I attempted to plot a graph. I don't know what the error was but it was fixed when I ran gretl from a Slackware 12.1 installation. After that, gretl plotting worked.
Tuesday, September 01, 2009
Slackware 13 Automount USB Drive
Scratch this: It worked once and now it doesn't work upon reboot!!!
This is so frustrating I may just give up on Slackware altogether.
With Slackware 12.1 and 12.2, USB drives automounted out of the box. Not so for my Slackware 13 installation. The obvious thing was to check which groups I was already a member of, using the command
groups
in Konsole. It said I was a member of cdrom, but not plugdev, so I used the command
gpasswd -a [user] plugdev
and logged out. I logged in again, double checked that I was in plugdev, and tested. Nothing. I didn't know what to do and Google didn't help. I added myself to a bunch of other groups, but none worked.
Some more extensive searching took me here:
http://www.linuxquestions.org/questions/slackware-14/12.0-and-hal-read-this-566862/
Long story short, I had to open the file /etc/login.defs as root and add plugdev to the line referenced in that post. I logged out and back in, and then I could automount USB drives.
This is so frustrating I may just give up on Slackware altogether.
With Slackware 12.1 and 12.2, USB drives automounted out of the box. Not so for my Slackware 13 installation. The obvious thing was to check which groups I was already a member of, using the command
groups
in Konsole. It said I was a member of cdrom, but not plugdev, so I used the command
gpasswd -a [user] plugdev
and logged out. I logged in again, double checked that I was in plugdev, and tested. Nothing. I didn't know what to do and Google didn't help. I added myself to a bunch of other groups, but none worked.
Some more extensive searching took me here:
http://www.linuxquestions.org/questions/slackware-14/12.0-and-hal-read-this-566862/
Long story short, I had to open the file /etc/login.defs as root and add plugdev to the line referenced in that post. I logged out and back in, and then I could automount USB drives.
Tuesday, July 14, 2009
Multivariate normal distribution in Python
I could not find a Python function to evaluate the multivariate normal distribution in Python. Here's one that gives equivalent results to the dmvnorm function in the mvtnorm package for R. It's something that works. I've not had time or need yet to fix it up.
b: A vector
mean: The mean of the elements in b (same dimensions as b)
cov: The covariance matrix of the multivariate normal distribution
License: GPL version 2
http://www.gnu.org/licenses/gpl-2.0.html
k = b.shape[0]
part1 = numpy.exp(-0.5*k*numpy.log(2*numpy.pi))
part2 = numpy.power(numpy.linalg.det(cov),-0.5)
dev = b-mean
part3 = numpy.exp(-0.5*numpy.dot(numpy.dot(dev.transpose(),numpy.linalg.inv(cov)),dev))
dmvnorm = part1*part2*part3
b: A vector
mean: The mean of the elements in b (same dimensions as b)
cov: The covariance matrix of the multivariate normal distribution
License: GPL version 2
http://www.gnu.org/licenses/gpl-2.0.html
k = b.shape[0]
part1 = numpy.exp(-0.5*k*numpy.log(2*numpy.pi))
part2 = numpy.power(numpy.linalg.det(cov),-0.5)
dev = b-mean
part3 = numpy.exp(-0.5*numpy.dot(numpy.dot(dev.transpose(),numpy.linalg.inv(cov)),dev))
dmvnorm = part1*part2*part3
Wednesday, July 01, 2009
Static IP address information in Slackware 12
To find information about a static IP address in Slackware 12:
1. Open /etc/rc.d/rc.inet1.conf
Find the IP address, gateway, and netmask.
2. Open /etc/resolv.conf
There will be information about the nameserver(s).
You can restart the network using the command
/etc/rc.d/rc.inet1 restart
This came in helpful, because I wanted to set up my static IP in Vector 6.0, needed to pull the information from my Slackware installation, but was unable to reboot my computer. I was able to enter the information using Netconf in the Control Center but did not have a way to restart the network.
1. Open /etc/rc.d/rc.inet1.conf
Find the IP address, gateway, and netmask.
2. Open /etc/resolv.conf
There will be information about the nameserver(s).
You can restart the network using the command
/etc/rc.d/rc.inet1 restart
This came in helpful, because I wanted to set up my static IP in Vector 6.0, needed to pull the information from my Slackware installation, but was unable to reboot my computer. I was able to enter the information using Netconf in the Control Center but did not have a way to restart the network.
Wednesday, September 05, 2007
R packages with ACML
My main production machine has a 64-bit AMD processor and runs Debian Lenny as the primary OS. As I have noticed substantial (often 5x) speed improvements, I always compile R myself against the ACML. For what I do, the difference is important.
I got the following error when trying to install some packages:
/usr/bin/ld: cannot find -lblas
collect2: ld returned 1 exit status
After Google didn't help, and I couldn't find anything explicit in the manuals about this error, I read through "A.3.1.5 Shared BLAS" in the Installation and Administration manual. After a little experimenting, I found that the solution is to make a symbolic link using
ln -s /opt/acml3.6.1/gfortran64_mp/lib/libacml_mp.so /usr/local/lib64/R/lib/libblas.so
AFAICS, the symbolic link needs to go in the same directory as libRlapack.so, if the directory structure happens to be different from my current directory structure. I assume that this will work with any version of ACML provided it has the .so extension.
What is different from anything in the manual is that the symbolic link needs to be a file named libblas.so, not libRblas.so. Note that the error said R could not find -lblas, which means it is looking for a file named libblas.so. Otherwise the error would have said -lRblas was not found.
I got the following error when trying to install some packages:
/usr/bin/ld: cannot find -lblas
collect2: ld returned 1 exit status
After Google didn't help, and I couldn't find anything explicit in the manuals about this error, I read through "A.3.1.5 Shared BLAS" in the Installation and Administration manual. After a little experimenting, I found that the solution is to make a symbolic link using
ln -s /opt/acml3.6.1/gfortran64_mp/lib/libacml_mp.so /usr/local/lib64/R/lib/libblas.so
AFAICS, the symbolic link needs to go in the same directory as libRlapack.so, if the directory structure happens to be different from my current directory structure. I assume that this will work with any version of ACML provided it has the .so extension.
What is different from anything in the manual is that the symbolic link needs to be a file named libblas.so, not libRblas.so. Note that the error said R could not find -lblas, which means it is looking for a file named libblas.so. Otherwise the error would have said -lRblas was not found.
Thursday, July 12, 2007
Using lapply in R
R is generally pretty good, although it has more quirks than other languages, such as automatically dropping dimensions from an array. (You have to specify drop=false every time, there is no way to change the default, apparently the R development team is taking cues from Microsoft.)
Anyway, buried in the documentation is the lapply function, which makes your code run faster when it applies. It can help you avoid having to link in compiled code, i.e., Fortran or C. Here is an example of it in use.
y <- function(x) { z <- x^3 }
x <- 3:5
b <- lapply(x, y)
w <- 1:6
z <- 4
b <- lapply(w, function(x) {
g <- x^3+z
h <- g*2
})
The first example applies the function y to each element in x. The second example applies the values in w to function(x). It shows that you can access global variables inside the function used inside lapply.
Anyway, buried in the documentation is the lapply function, which makes your code run faster when it applies. It can help you avoid having to link in compiled code, i.e., Fortran or C. Here is an example of it in use.
y <- function(x) { z <- x^3 }
x <- 3:5
b <- lapply(x, y)
w <- 1:6
z <- 4
b <- lapply(w, function(x) {
g <- x^3+z
h <- g*2
})
The first example applies the function y to each element in x. The second example applies the values in w to function(x). It shows that you can access global variables inside the function used inside lapply.
Subscribe to:
Posts (Atom)