Thanksgiving

"The miraculous thing about this country is that almost everybody has food, clothing, shelter, and extraordinary devices undreamed of until a moment ago in human history: radios, telephones, color televisions, cars, radios in their cars even enough dough to fly across the country once a year, if they plan ahead and stay over a Saturday. I speak here not just of the great middle class." Andrew Tobias, 1996

Andrew talks about the material possessions, of which food, clothing, and shelter can only be taken for granted by most people in this country, but I'm certainly grateful for them. Working in the healthcare field, I'm thankful for my health. Also, my family and friends.

But what I'm mostly thankful for today is free software. Not just because it costs less, but the breadth and quality of open source software makes it a joy to work with computers. So thanks to Guido van Rossum, Linus Torvalds, and all the others who have dedicated contributed to free, high-quality software.

---

PostgreSQL is pretty cool. I'm now starting to use it in favor of MySQL. But I spent almost an hour today figuring out how to retrieve column names from a table. Resulting SQL:

  SELECT attname FROM pg_class, pg_attribute WHERE relname='billcode' and 
    pg_class.oid=attrelid and attnum > 0 ORDER BY attnum;

Anyone know of a better way?

jEdit

A couple people whose opinions I respect have expressed admiration for jEdit, a programmer's editor written in Java.

Since Java is non-free software, it requires more than a simple apt-get command to install jEdit on my machine. Fortunately, the Ubuntu How-To lists a fairly straightforward procedure to install the Java runtime. I used to write quite a bit of code in Java. It's interesting that I haven't had Java installed on my development machine in some time.

A few minor tweaks later and jEdit is running. It supports an large number of features, plug-ins, language support, etc. You move the cursor around with the standard arrows, Home/End, PgUp/PgDn keys, which will make it easy for most Windows users. As a touch typist, however, I prefer to keep my fingers on the keyboard - even moving my right hand to the arrow keys disrupts my speed and concentration. It's certainly possible to remap keys to control-key combinations, but I dislike building special-purpose key configurations. It's always a problem, when you move to another machine, even temporarily. The final negative is the memory required for jEdit. For comparison:

• vi, Vim: 4.7MB


• xemacs: 19.4MB


• jEdit: 258MB

Memory is getting cheaper, but I can run a dozen emacs sessions (or 50 vi sessions) for the cost of one running jEdit process. I think this is the showstopper for me, though I'll continue to play with jEdit.

The high price of low-bandwidth data

The current situation with mobile phone communications is crazy. In the U.S., it's relatively easy to send a short text message (SMS) to someone's cell phone via email, e.g.:

• T-Mobile: phonenumber@tmomail.net
  
• Cingular: phonenumber@cingularme.com
  
• Sprint: phonenumber@messaging.sprintpcs.com
  
• Verizon: phonenumber@vtext.com
  
• Nextel: phonenumber@messaging.nextel.com
  

Apparently in Europe the carriers make sms-to-email more difficult.

However in Euroland, they make it relatively easy to send outbound SMS messages from your cell phone to internet email accounts. In the U.S., I couldn't find a single provider for sms-to-email delivery. T-Mobile will allow me to send email (not SMS) to the internet, but I must first setup a POP server. Since all I want is to send simple outbound messages via Python code, this seems like overkill. I don't want or need inbound email.

So my current round-trip scenario:

• server-to-phone: send automated email to phonenumber@tmomail.net
  
• phone-to-server: send sms via connectotel
  

Connectotel is an free offshore sms-to-email gateway service. Free is a bit of a misnomer, however, as each overseas message costs me $0.15. Even domestic messages cost $0.05 each.

There's something quite screwy in that my voicemail, which consumes orders of magnitude more transmission and storage, is free. I could easily spend over $50/month in SMS fees for less than 12K of data.

Most of the time this is a non-issue since I'll be using my hacked bluetooth server for local data communications.

svk vs. svn

Fredik Lundh says "If you're using subversion for anything, you really want to use svk as a front-end, instead of svn." I installed version 1.00 of svk via apt-get.

svk is written in Perl and I notice the command line response seems very sluggish.

After creating a respository, I attempted to import a project. No response. A query to #svk chat confirms that it's failing silently. The recommendation is that I upgrade to version 1.05, but I think I'll just pass for now. Or maybe just just svk as a client for Subversion.

Upgrading to Ubuntu Breezy / XEmacs

It's a rainy Sunday, a good time to upgrade my Ubuntu desktop from Hoary 5.04 to Breezy 5.10. A few hours later I'm frustrated. The upgrade process gives me errors which I spend too much time resolving. I have backups of /etc and /home/jbauer, but a system restore would not have been pleasant. I won't go into the details, but enough time spent on Ubuntu forums found the answers to most of my questions. Upgrading is a task for the steel-willed, not nervous nellies such as myself.

I prefer XEmacs to GNU/Emacs, though I could live with either. One problem with running XEmacs from the command line is the annoying messages:

Warning: Missing charsets in String to FontSet conversion
Warning: Missing charsets in String to FontSet conversion
Warning: Unable to load any usable fontset

My LANG environment is en_US.UTF-8, so my short-term fix is to create a command line script, ~/bin/xe:

LANG=C xemacs $* &

Success with SWIG

It bothered me yesterday that I didn't get a Bluetooth server running with Python, so I had to go back to SWIG. The Pyrex framework is probably better for major projects, but SWIG wins out for write-once applications that won't require a lot of maintenance.

My swig file, pyobexserver.i:

    %module pyobexserver
    %{
    #include "openobex/obex.h"
    %}
    int get_my_file(char *outfile);

From the Python interpreter:

    >>> from pyobexserver import get_my_file
    >>> get_my_file('/home/jbauer/nokia/files')
    Waiting for connection...

What could be easier? From downloading SWIG to running a functional program: 30 minutes. I love Python.

Subversion: status vs. update

Today I learned something important about svn status the hard way. Prior to this project, I've only used the rudiments of Subversion. For most SourceForge projects, I would run svn update. The svn status command displays all files that have been modified locally. I assumed it compared changes against the repository. So yesterday when I tried to commit some changes, I got an "Out of date" error message. I had to copy my mods elsewhere, update, manually diff the files, then commit. No big deal. I just didn't read the documentation closely enough.

Gnome: How do I get the properties of a program listed in my applications menu? Most windowing systems do this with a right mouse click. It's not obvious with gnome, so my workaround is to install the application on my panel, right-click, then remove it from the panel. This is how I discovered the "Bluetooth File Sharing" application is actually gnome-obex-server.

Speaking of gnome-obex-server (written in Python) and obexserver (written in C) -- they are incredibly limited pieces of software. They both accept incoming files from bluetooth devices -- in my case pictures from my Nokia 6600 -- but give no option to choose where you want them saved. The gnome-obex-server drops the files in your home directory, and obexserver puts everything in /tmp. Another issue with obexserver -- it shuts down after it has received the first incoming file. It's a simple matter to modify the code in obexserver, but both of these are Debian distribution apps -- obexserver even has a man page.

I looked into using Pyrex to build a Python interface to the OBEX stuff. Pyrex is an awesome piece of work, but I don't have time to wrap up all the OBEX C structures in a .pyx file. There's a bit of a learning curve to get it right. Since the interface is so simple, I might SWIG it instead. I shouldn't be wasting time on this anyway.

YAML vs. JSON

I have some information that doesn't quite fit into a relational database model, though it would be convenient to store it as such. Here's an example:

>>> data = {'therapies': ['4'], 'inpatient_diagnoses': [('751.1', 'ATRESIA SMALL INTESTINE')], 'prior_conditions': ['3'], 'diagnoses_and_severity': [('879.2', 'OPN WND ANTERIOR ABDOMEN', '2'), ('751.1', 'ATRESIA SMALL INTESTINE', '4'), ('751.2', 'ATRESIA LARGE INTESTINE', '2')], 'race': ['6'], 'regimen_changed': '1', 'current_payment_sources': ['1'], 'overall_prognosis': '1', 'high_risk_factors': ['2','5'], 'inpatient_discharge_date': 2005-02-19, 'financial_factors': ['0'], 'rehab_prognosis': '1', 'inpatient_facilities': ['1'], 'changed_diagnoses': [('751.1', 'ATRESIA SMALL INTESTINE')], 'life_expectancy': '0'}

It's a combination of lists, strings, and associative arrays, supported by most modern languages. One goal is to keep the data human-readable, which precludes some kinds of binary serialization. The data could be represented as XML, but XML's just barely readable.

Matt Goodall offered the following suggestions:

• YAML Ain't Markup Language (tm)
  
• JSON - JavaScript Object Notation

I recall playing with the Python version of YAML over a year ago. It was pretty cool, though slow, and not a complete implementation. There's a certain appeal to a Python programmer in representing data with whitespace. For example, an invoice:

invoice: 34843
date   : 2001-01-23
bill-to: &id001
    given  : Chris
    family : Dumars
    address:
        lines: |
            458 Walkman Dr.
            Suite #292
        city    : Royal Oak
        state   : MI
        postal  : 48046
ship-to: *id001
product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00
tax  : 251.42
total: 4443.52
comments:
    Late afternoon is best.
    Backup contact is Nancy
    Billsmer @ 338-4338.

I couldn't google up an emacs mode for .yml files, but I'm sure someone must have written one by now. Ultimately though, I'd prefer to keep embedded newlines out of the field data, so I next looked at JSON.

>>> import json
>>> s = json.write(data)  # data is returned as a string.
>>> d = json.read(s)      # converted back into a dictionary.

Pretty simple.

JSON wins out over YAML for this project. It was dirt simple to round-trip the example data. Speed isn't much of an issue for this particular use of JSON and I like storing the data as TEXT rather than BLOB.

There is also JSON-RPC, a remote procedure call protocol similar to XML-RPC. Interesting.

Trac: Integrated SCM & Project Management

After working on my current project I have a better
appreciation for having issue tracking, a wiki, and the
Subversion revision control system integrated into
a complete package.

If you're already familiar with cvs or Subversion, Trac
is useful without having to learn a bunch of arcane
commands. It would be fairly easy for even a non-programmer
to participating in a project managed with Trac.

The Trac wiki isn't as robust as MoinMoin, but the
effortless integration with the ticket tracker
is a pleasure to use. Project members can
be issued notifications via email or RSS for tickets
that have issued, modified, or resolved.

The source browser is also well integerated. The
current version of each code module has a link to
the revision, which in turn displays all the other
files that were committed as part of the revision.

Python for Series 60

Yesterday my Dad loaned me his USB bluetooth dongle. I put it on sulla, a linux box running Ubuntu 5.04. After the usual frustrations, I got my Nokia 6600 to communicate, sending files in each direction.

At last year's PyCon I met Jukka Laurila and Erik Smartt from Nokia who were demonstrating Python for the series 60 phones.Ever since then, I've been wanting to get Python running on my nokia 6600. With some help from the #pys60 channel, I was able to run the interpreter on the phone via bluetooth. Pretty cool.

I spent too much time with bluetooth and not enough on learning Nevow. Today was unseasonably warm, so I got out on the bike for a couple hours despite wanting to spend more time on development projects.