python builtin inflation

There's some talk on python-dev about adding defaultdict. One consequence of tinkering with Python is the number of builtins increase over time. Here's my only-slightly-tongue-in-cheek proposal. The total amount of builtins should be limited to a standard sized 80x24 xterm screen:

Python 2.4.2 (#2, Sep 30 2005, 21:19:01)
>>> dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FloatingPointError', 'FutureWarning', 'IOError', 'ImportError', 'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'OverflowWarning', 'PendingDeprecationWarning', 'ReferenceError', 'RuntimeError', 'RuntimeWarning', 'StandardError', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__debug__', '__doc__', '__import__', '__name__', 'abs', 'apply', 'basestring', 'bool', 'buffer', 'callable', 'chr', 'classmethod', 'cmp', 'coerce', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'execfile', 'exit', 'file', 'filter', 'float', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'min', 'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range', 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'unichr', 'unicode', 'vars', 'xrange', 'zip']
>>>

We've got a little room for futher expansion, but many more will have to come at the cost of removing others. How did 'zip' ever get in there? ;-)

Interesting to compare with "classic" Python 1.52 ...

>>> dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'FloatingPointError', 'IOError', 'ImportError', 'IndexError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplementedError', 'OSError', 'OverflowError', 'RuntimeError', 'StandardError', 'SyntaxError', 'SystemError', 'SystemExit', 'TypeError', 'ValueError', 'ZeroDivisionError', '__debug__', '__doc__', '__import__', '__name__', 'abs', 'apply', 'buffer', 'callable', 'chr', 'cmp', 'coerce', 'compile', 'complex', 'delattr', 'dir', 'divmod', 'eval', 'execfile', 'exit', 'filter', 'float', 'getattr', 'globals', 'hasattr', 'hash', 'hex', 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'len', 'list', 'locals', 'long', 'map', 'max', 'min', 'oct', 'open', 'ord', 'pow', 'quit', 'range', 'raw_input', 'reduce', 'reload', 'repr', 'round', 'setattr', 'slice', 'str', 'tuple', 'type', 'vars', 'xrange']

It's not all fun and games

I'm wearing my sysadmin hat today. Last night
I started an upgrade for trajan (Debian Sarge).
When I got in early this morning it wouldn't
boot, just hung at "LI". The usual panicky
minutes before verifying that the nightly
backups completed.

Next, attempt to boot up on Knoppix
and verify the filesystems are okay.

I attempted to rebuild LILO (yes, I use GRUB
for all the new installs):

# mount /dev/hda3 /mnt/hda3
# chroot /mnt/hda3
# /sbin/lilo

But Knoppix won't write lilo to /dev/hda,
even after I've given it write access. This
is one of the few times Knoppix has let me
down. Fortunately, I've got another Linux
Rescue CD with the 2.4 kernel. Booting up
on it finally allows me to rebuild LILO. My
decision process was much more time consuming
than appears above, but that was the general
flow. Stupid Jeff: I didn't have a rescue
diskette built, at least not one that was
readily available. A boot diskette would have
saved valuable time.

Unfortunately there are now some residual
problems from the upgrade. Python spews some
wierd linkage warnings (fixed by recompiling
the source), but Apache won't allow any users
to log on.

An awful lot of cursing/head-scratching later
reveals a couple of very obscure Apache bugs:

The gist of the matter is that I must swap the
order in which the following modules appear in
the configuration.

LoadModule auth_module /usr/lib/apache/1.3/mod_auth_ssl.so
LoadModule apache_ssl_module /usr/lib/apache/1.3/libssl.so

Also, I had to turn off SSLFakeBasicAuth, which
was on by default.

Quite a nerve-wracking morning.

Twisted online API documentation is hopeless

There's an outstanding book on Twisted by Abe Fettig. I've been meaning to get down into the details of Twisted rather than devote all my time with Nevow. The examples are clear and well-written, e.g.:

from twisted.internet import reactor
import time

def printTime():
    print "current time is", time.strftime("%H:%M:%S")

def stopReactor():
    print "stopping reactor"
    reactor.stop()

if __name__ == '__main__':
    reactor.callLater(1, printTime)
    reactor.callLater(2, printTime)
    reactor.callLater(3, printTime)
    reactor.callLater(4, printTime)
    reactor.callLater(5, stopReactor)
    print "running the reactor ..."
    reactor.run()
    print "reactor stopped"

However, I next go to the twisted website to review the reference API. Guess what? There's no entry tor twisted.internet.reactor. Reading the source reveals that selectreactor is the default implementation of reactor. It's pretty obvious that the parameter to the callLater method are time (in seconds) and and argument-less function, but I'd like to review that in the API documentation.

The twisted.internet.selectreactor section lists no callLater method. Grepping through the source reveals that it's implemented in four places, but the one I'm probably using is twisted.internet.base. There's a reference in the docstring: "See twisted.internet.interfaces.IReactorTime.callLater."

Okay, I can live with this, but it means the online API is a waste of time. I'll simply have to read through the source code to learn anything useful about twisted. The online API actually slows down the discovery process.

Update: Here's a line of Nevow code:

request = inevow.IRequest(ctx)

Want know to what IRequest returns? Check out the API docs.

Acroread on Unix

Adobe's acroread is another non-GPL piece of software that I find useful for software development, since many specifications are now distributed in pdf format. Unfortunately, too many spec sheets are still tied to the proprietary Microsoft Word format, for which the only remedy is OpenOffice.org or some of the various lightweight MS-Word readers.

Acroread is a bit heavyweight for most viewing tasks, which is why xpdf is still my default pdf viewer. But for certain kinds of documentation, such as 700+ page HIPAA implementation guides, having acroread is indispensible with it's window tiles to aid navigation.

Having acroread installed is useful for software development with *
ReportLab. It's handy to view the output of generated pdf files with two different renderers, though I also check the results on Windows as well.

Acroread on Unix is much improved. For a while it appeared that the Linux/Solaris versions were languishing behind the Windows versions. Unix acroread has one feature not available on the Windows version. It can emit the Postscript to a file or stdout - useful for batch print jobs.

Something new I just learned: Compressed pdf files can be viewed with the zxpdf command.

remote backups with Strongspace.com

I've been wanting to use an off-site backup process that was simple to administer, yet easy for non-techies to use. Strongspace.com appears to fit the bill. You basically sign up for a particular plan that gives you a specific amount of disk space and a number of users. There's no signup fee and you can discontinue the service at any time.

Strongspace works with sftp/scp and rsync+ssh. For windows users, it can be accessed via WinScp or Filezilla. But for making convenient backups from Windows machines nothing beats an rsync batch file linked to a desktop icon. Even better, would be to schedule automatic backups. The advantage with rsync is that after the initial backup, updates are generally very quick.

The disadvantage with rsync is that it's not an easy program for Windows users to install. Fortunately, David Barrett has written an excellent How-To.

A typical rsync batch file entry will look like this:

rsync -avz "/Documents and Settings/user/My Documents" myaccount@mydomain.strongspace.com:Documents

Backups are only half the story, however. Eventually, there will come a day when the user needs to retrieve lost data. This is where having a browseable backup site is handy. For the user, retrieving a lost document is as simple as accessing her account on Strongspace.com. No intervention from me is necessary. Furthermore, the data can be retrieved from any other location, e.g. modifying a contract or spreadsheet while off-site. It would certainly be possible to configure something like this internally, but it would be a hassle to maintain with limited advantages.

Thanksgiving

"The miraculous thing about this country is that almost everybody has food, clothing, shelter, and extraordinary devices undreamed of until a moment ago in human history: radios, telephones, color televisions, cars, radios in their cars even enough dough to fly across the country once a year, if they plan ahead and stay over a Saturday. I speak here not just of the great middle class." Andrew Tobias, 1996

Andrew talks about the material possessions, of which food, clothing, and shelter can only be taken for granted by most people in this country, but I'm certainly grateful for them. Working in the healthcare field, I'm thankful for my health. Also, my family and friends.

But what I'm mostly thankful for today is free software. Not just because it costs less, but the breadth and quality of open source software makes it a joy to work with computers. So thanks to Guido van Rossum, Linus Torvalds, and all the others who have dedicated contributed to free, high-quality software.

---

PostgreSQL is pretty cool. I'm now starting to use it in favor of MySQL. But I spent almost an hour today figuring out how to retrieve column names from a table. Resulting SQL:

  SELECT attname FROM pg_class, pg_attribute WHERE relname='billcode' and 
    pg_class.oid=attrelid and attnum > 0 ORDER BY attnum;

Anyone know of a better way?

jEdit

A couple people whose opinions I respect have expressed admiration for jEdit, a programmer's editor written in Java.

Since Java is non-free software, it requires more than a simple apt-get command to install jEdit on my machine. Fortunately, the Ubuntu How-To lists a fairly straightforward procedure to install the Java runtime. I used to write quite a bit of code in Java. It's interesting that I haven't had Java installed on my development machine in some time.

A few minor tweaks later and jEdit is running. It supports an large number of features, plug-ins, language support, etc. You move the cursor around with the standard arrows, Home/End, PgUp/PgDn keys, which will make it easy for most Windows users. As a touch typist, however, I prefer to keep my fingers on the keyboard - even moving my right hand to the arrow keys disrupts my speed and concentration. It's certainly possible to remap keys to control-key combinations, but I dislike building special-purpose key configurations. It's always a problem, when you move to another machine, even temporarily. The final negative is the memory required for jEdit. For comparison:

• vi, Vim: 4.7MB


• xemacs: 19.4MB


• jEdit: 258MB

Memory is getting cheaper, but I can run a dozen emacs sessions (or 50 vi sessions) for the cost of one running jEdit process. I think this is the showstopper for me, though I'll continue to play with jEdit.