Unix: Compiling that code

Several emails from readers complain that some of the newer applications I mention in these columns aren't available as pre-packaged binaries, and the only way to get the latest version is to download the source code and compile it yourself. I gather that some of you don't feel up to this, while others who have bravely tried have run into insuperable obstacles.

Don't panic! If you haven't tried compiling before, my advice is simply to give it a go. There will be plenty of time for panic afterwards.

Why compile? Because the ability to turn source code into binaries on your own system is one of the single most powerful features of a fully-fledged Unix system. It's the difference between being a couch-potato living on junk food, and a gourmet with a fully equipped kitchen.

In fact, with a free software Unix implementation like GNU/Linux, I'm inclined to put on my hardcore hacker's hat and argue that it should probably be seen as the standard way of installing new software, with RPM or DEB binary packages as the fallback.

Incidentally, I'm not a hardcore hacker myself - I just have the hat - but I hope you'll see my point when you've read what follows.

Talking about tarballs

Binary packaging systems do a couple of really useful things: they ensure that all the dependencies (this binary can't run without these libraries) are in place, and they make it as easy to take applications off the system as to put them on.

They're also capable of running pre-installation and post-installation scripts that prepare the ground for the installation and clean-up afterwards. But properly organised source tarballs can do all these things too.

I'll admit that many of the source tarballs you're likely to come across will be in a fairly ragged state. This isn't because the authors are lazy, or out to get you. In the white heat of development there may not be time to add refinements like configure - the author just wants to get the code out there so others can test it.

If you're not keen to be at the cutting edge, it might be a good idea to avoid this kind of source code.

However, best-case source tarballs can be almost as easy to install as their binary package equivalents. They'll take a little or a lot longer, because the source has to be turned into binary on your system.

But their huge advantage is that each source package can be cross-platformed. This means it will work for many different operating systems using the same processor, or even across different processors.

All about autoconfig

During the 30-year history of Unix there have been many attempts at achieving this nirvana of cross-platform packaging. But arguably the most widely used today, and easily the best, is the GNU programming utility autoconfig, written by David MacKenzie of the Free Software Foundation (FSF).

In the early 1990s MacKenzie was responsible for testing the source code for the GNU suite of software tools across multiple platforms. He began by writing a configure script for each tool.

The script would know the requirements of the source code for that particular tool, and would test the environment it was running in to see what resources were available on that specific platform. Then it created compiler configuration scripts that matched these requirements and resources.

A lot of manual cutting and pasting went on as MacKenzie generated scripts for each tool. The sensible thing, surely, he thought, would be to delegate this job to the computer.

Thus began what MacKenzie calls, in his quasi-Biblical chronicle of the project, "the journey from the slavery of hand-written configure scripts to the abundance and ease of Autoconf" (see the History chapter at www.gnu.org/manual/autoconf).

Producing shell scripts

Autoconf is a tool the source code author can use for producing special shell scripts that will be bundled with the distribution tarball. When these scripts are run on the target machine (the end user's own computer), they will automatically configure the source code packages to adapt to the operating system that's running and the hardware it's running on.

The generated scripts are designed to work across a whole variety of Unix-like systems, which can even include Windows systems if they have the full Cygwin environment http://sources.redhat.com/cygwin).

I download all my non-kernel source code tarballs into a directory called /usr/src/other. I can run a quick check to see whether there are any 'autoconfiscated' (MacKenzie's word for it) tarballs in there by changing to this directory and running this one-liner:

$ for A in *.tgz *.tar.

gz ; do tar tzf $A - grep

configure.in ; done

(Note: code string reads as one line)

tgz and tar.gz are the two most common suffixes for tarballs, so what we're doing here is taking all the filenames that fit this description ($A) and running tar's list (t) function on them to see what files each tarball contains. The z is necessary here because the tarballs have been compressed.

Then we pipe the output of that through the search tool, grep, looking for a file named configure.in, which is the special config file generated by Autoconf. Its presence is a pretty sure indicator that the package has been autoconfiscated.

Three steps to binary heaven

Those are the basics of Autoconfig. This is the point where you might expect me to break off and say something like "next month, we'll walk through some actual examples". But I won't keep you in suspense - from the target machine's point of view, the thing is so simple that we can dispose of the subject in three very short steps:

1) Untar the tarball (tar xzf ) and change into the resulting directory.

2) Type ./configure and watch the script check out your system, printing a one-line message for each check "so the user doesn't get too bored" (MacKenzie).

3) When that's done, type make. The compiling system will read the config files generated in step 2 and create your binaries.

Job done. For convenience, there may be an optional make test you can do to ensure the binaries are working properly, followed by make install, which will place the executables in the proper directories for your system.

THE CREATIVE DAP

OK, I confess. The other day I installed Microsoft Windows 98 on an old laptop of mine. The nice people at Creative have sent me one of their gorgeous MP3 Digital Audio Players (DAP) in connection with some other stuff I'm writing. The bad news is that it only works with Windows and Mac software.

A couple of months ago I mentioned I was exploring the idea of covering MacOS X here, as it's a Unix variant, but I've heard nothing from Apple since. So Windows it had to be.

The DAP (silly name; elsewhere in the world it's called the Nomad Jukebox, or NJB for short) has to interoperate with my Unix network.

The idea is that the MP3s originate from my CD collection, ripped using Joerg Schilling's excellent cdrecord utility on one of my Linux boxes, then go through the Windows 98 laptop and onto the DAP through a USB connector.

As I can't connect the DAP directly into the Unix network, I started to view the DAP/Win98 laptop combination as a single external unit. Call it the DapLap.

The Samba connection

Looked at in this light, the DapLap is eminently connectable, thanks to Samba, the free software SMB file server www.samba.org).

Samba lets me use Windows' own SMB filesharing to share directories on one of my Linux servers with the Windows 98 laptop. The wavs I rip go straight to the Samba server, and once they're there, I use the lame command line utility http://lame.sourceforge.net) under Linux to crunch them down to 192Kbits/sec MP3s.

From that server I can play them on any MP3 player on the network. Or, of course, pick them up from the DapLap to create music on the move.

The VNC remote display system

Smart readers may notice I've cheated slightly in introducing the concept of the DapLap. To transfer files between the network and the DAP, I still need to be working with the Windows 95 interface. Only if I can do this from anywhere on the network does the DapLap make sense as a single entity.

Happily, though, I can do just this with VNC. We've talked before about this free software from www.uk.research.att.com/vnc, but I'll remind you that VNC (or Virtual Network Computing) is a remote display system which (in this case) can take the whole of my Windows 98 desktop and display it on any other machine on the network that is running VNC client software.

Software development for DAP

Even so, it's a serious downer that there's no Linux software for the DAP, and James Ball, Creative UK's marketing manager, told me the company has no plans to develop any.

But does this matter? In the case of gadgets like the Palm Pilot, the free software community has proved lively and ingenious enough to produce its own software, which sometimes can be more flexible and better than the commercial product. But to do this, programmers need information on the interfaces.

Earlier this year Creative launched a Windows-based Software Development Kit (SDK) for the DAP. A kit like this will typically include implicit, and perhaps explicit, information on the interfaces, and possibly for this reason Creative made SDK recipients sign a non-disclosure agreement.

Despite this setback, a free software initiative aimed at the DAP has got under way www.aracnet.com/~seagull/NJB), as an extension of the LibPDMP effort www.pointless.net/libpdmp).

That's the good news. Now for the very good news. The buzz in the newsgroups as I write this is that Creative has very wisely decided to scrap the non-disclosure requirement for its SDK. So, we can expect the interface information it contains to be filtering down through the free software development community very soon.

/b>

Robb Maddocks (robbmaddocks@hotmail.com) writes: "I've just installed linux4win (Mandrake 7.2) on my PC. When I write a script and make it executable using chmod 700 filename, I get an error message saying 'command not found', just after the command prompt. However, the same example works at university but not on my PC. I'd be grateful for any suggestions."

This one can be baffling. Using chmod 700 to set the file permissions will make the file executable (and read-writeable) for the owner only. So the first question we have to ask is, are you running it as the owner?

I presume the answer to that is yes, because otherwise you would see "Permission denied" on a typical Unix system.

So the second question is - and this is a twist that frequently fools people coming to Unix from Windows - "How can my script not be found, when I've just created it, and I can clearly see it sitting in my directory?"

The answer is that the shell that is executing the script will look through the directories listed in the $PATH environment variable to find your script. Some people like to include "." in this path (meaning "the current directory", whichever that may be).

But this isn't secure, and most distros don't do this by default.

In this case, the current directory won't be searched for your script, so it'll be "not found". If this is what's happening, you need to type ./ instead of just , to tell the shell exactly where to find it.

Another possibility is that it's something other than the executable script itself that is 'not found'. If it's a shell script, it might begin with a "hash-bang" line that looks like this:

(hash)!/bin/bash

This means "invoke bash and use it to run the rest of the script that follows". But what if bash isn't on your system, or is, but isn't in the /bin directory?

Similar remarks apply to any other tools or utilities your script may be trying to invoke. Are they all there?

The full version of this article, with screenshots, can be found on page 226 of the July 2001 issue of Personal Computer World.

Contact
Chris Bidmead at unix@pcw.co.uk. Please do not send unsolicited file attachments.