Get There by Cycle!
I just got done installing Python 2.5 and converting the trip planner code to use it instead of 2.4. The only required change was pointing the “dispatch” file at the 2.5 executable. No other changes to the code were required to make it run on 2.5.
A few little things also got cleaned up, and I updated to the latest versions of MySQLdb (1.2.2b3) and simplejson (1.5).
Hopefully nothing will blow up.
The development version of the Trip Planner uses a Postgres/PostGIS backend (instead of MySQL) SQLAlchemy as the ORM (instead of raw SQL), and PCL for Python geometry types (instead of our own ugly hacked versions).
Question: How do you move geometries out of Postgres/PostGIS into PCL types via SQLAlchemy and vice versa?
Answer: Create a custom geometry column type.
Here’s our SQLAlchemy geometry type definition and subtypes (points, linestrings, and multilinestrings; adding other types should be trivial):
Look here for an example of using it:
The type is pretty simple. The only tricky part was figuring out how the database stores the geometry. It’s in ASCII hex, so we use binascii.a2b_hex to get a binary representation and feed that to the PCL fromWKB factory. In the other direction we use binascii.b2a_hex on the WKB representation of the PCL geometry.
[8/7/09: Updated links to source code. Note: latest version uses Shapely, not PCL.]
I just tried out some o’ that new-fangle “Test Driven Development” (TDD) I’ve been hearing about. Yeah, it’s good stuff.
At the moment, I’m in the process of migrating GIS data from MySQL to PostgreSQL so we can take advantage of the PostGIS spatial extensions. I’ve also been making a bunch of related changes (AKA refactoring) in the “model”, separating things that never belonged together, and so forth.
I started out by rewriting the MySQL data import script. [Note: basically, the script pulls data out of a flat ESRI shapefile and normalizes it.] This wasn’t a complete, from-scratch rewrite–a lot of stuff I just copied over and tweaked a little bit. The biggest changes here were due to using SQLAlchemy instead of typing out SQL queries. I’ll just note that SQLAlchemy is “da bomb” and makes many things easier (once you get the hang of it).
That part was pretty straightforward, and low-level–I didn’t get into the ORM aspects of SQLAlchemy at all.
The next step was to modify the routine that creates adjacency matrices for routing. In the end, this was straightforward too. I ended up reusing some stuff from the new import script, which was cool. I refactored a lot during this process, adding some new modules and classes.
[Here's where we get to the TDD aspect.]
So, I was sitting there (here really) thinking, “Hmmm… what now?” I drew some diagrams with the new classes and associations…. OK, that’s fun…. “Wait, I know. Run the unit tests!” Doy!
I started with the address normalization service, since the other services both depend on it. The test suite for this service isn’t as comprehensive as it probably should be (there are 19 tests), but it proved to very useful for shaking out a bunch of bugs in all that refactoring. The tests also helped keep me focused, and that aspect might be more important than the bug-squashing aspect (maybe).
Today, address normalization. Tomorrow, geocoding.