Python Cartographic Library (PCL) — Installing Just the Spatial Package

Posted on by
2

So let’s say you need a spatial geometry library for Python. You could write your own; you could also use the PCL. The PCL includes some packages we don’t need, like one for MapServer rendering. I only installed the minimum needed to get the spatial package working (which I’ll talk about below).

[Note: "geometry" refers to points, lines, polygons and other geometric forms used to represent real-world objects. Examples: intersection (point), street (line), zip code boundary (polygon).]

I wrote a rudimentary geometry library for the trip planner that’s been working fine, but now I need to do some more “advanced” stuff related to using PostGIS and SQLAlchemy. In particular, I want to convert database values to Python objects and vice versa.

The first part (database to Python) is fairly easy and our current library already does that, but it’s convoluted in that it gets the database value as well-known text (WKT), parses that, and creates a Python object. From what I can tell, the PCL can go straight from well-known binary (WKB) geometry to Python objects.

The second part (Python to database) is harder because it involves converting a Python object to a binary geometry value. I don’t know anything about the binary geometry format and I don’t want to know, and it looks like with the PCL I don’t need to know.

I’m assuming PostGIS and PCL will get along together because they both rely on the same libraries, proj4 and geos. We’ll see.

The installation was fairly straightforward. The PCL includes five sub-packages. We had to install two of them, PCL-Referencing and PCL-Spatial. PCL-Referencing requires proj4. PCL-Spatial requires PCL-Referencing and geos >= 2.2.2. Something in there also requires the OGR library, which is included with GDAL.

The basic steps are, install proj, geos, and gdal, then install PCL-Referencing, and lastly install PCL-Spatial. On Ubuntu 6.06 (Dapper), here’s what I actually did:

  • Installed proj4 using apt-get
  • Installed libgdal using apt-get
  • Installed geos 2.2.3 from source into /usr/lib. I installed this over a slightly older version of geos installed using apt-get; hopefully that won’t cause any issues.
  • Checked out the PCL trunk:
    svn co http://svn.gispython.org/gispy/PCL/trunk PCL
  • Installed PCL-Referencing with the usual python setup.py install
  • Installed PCL-Spatial with the usual…

Revolver Bikes and North Portland Bike Works Have byCycle Shirts

Posted on by
1

Now you can support local North Portland bike shops AND byCycle.

I pedaled up to Revolver Bikes last week to deliver shirts and check out the shop. Revolver is in a great location – next to New Seasons on Interstate and right on the Max line. Mark was very friendly, relaxed, and although busy, did not seem rushed. I think this might be one of my favorite bike shops in town.

Revolver Bikes has three grey track Shirts – one of each size, a cut out shirt and dress.

North Portland Bike Works has a selection of track shirts, mostly large sizes.

If you have a shop and what to carry our shirts, email Lauren.

You can still buy shirts from us here.

Test Driven Development, PostgreSQL, SQLAlchemy

Posted on by
1

I just tried out some o’ that new-fangle “Test Driven Development” (TDD) I’ve been hearing about. Yeah, it’s good stuff.

At the moment, I’m in the process of migrating GIS data from MySQL to PostgreSQL so we can take advantage of the PostGIS spatial extensions. I’ve also been making a bunch of related changes (AKA refactoring) in the “model”, separating things that never belonged together, and so forth.

I started out by rewriting the MySQL data import script. [Note: basically, the script pulls data out of a flat ESRI shapefile and normalizes it.] This wasn’t a complete, from-scratch rewrite–a lot of stuff I just copied over and tweaked a little bit. The biggest changes here were due to using SQLAlchemy instead of typing out SQL queries. I’ll just note that SQLAlchemy is “da bomb” and makes many things easier (once you get the hang of it).

That part was pretty straightforward, and low-level–I didn’t get into the ORM aspects of SQLAlchemy at all.

The next step was to modify the routine that creates adjacency matrices for routing. In the end, this was straightforward too. I ended up reusing some stuff from the new import script, which was cool. I refactored a lot during this process, adding some new modules and classes.

[Here's where we get to the TDD aspect.]

So, I was sitting there (here really) thinking, “Hmmm… what now?” I drew some diagrams with the new classes and associations…. OK, that’s fun…. “Wait, I know. Run the unit tests!” Doy!

I started with the address normalization service, since the other services both depend on it. The test suite for this service isn’t as comprehensive as it probably should be (there are 19 tests), but it proved to very useful for shaking out a bunch of bugs in all that refactoring. The tests also helped keep me focused, and that aspect might be more important than the bug-squashing aspect (maybe).

Today, address normalization. Tomorrow, geocoding.

FreekBox

Posted on by
Reply

Today we went to Free Geek and picked up a Grant Box. It’s a pretty sweet computer, especially for the price ($0). We received the box through their hardware grant program. Here’s a little about Free Geek from their mission page:

“FREE GEEK is a 501(c)(3) not for profit community organization that recycles used technology to provide computers, education, internet access and job skills training to those in need in exchange for community service.”

The box came with Ubuntu GNU/Linux pre-installed, which is excellent because that’s what I’m running on my development box and on our new virtual private server (which now hosts our project tracker and soon will host a new version of the trip planner).

Our plan for the box is to use it for testing purposes (e.g., browser compatibility) and/or as a staging server before pushing new versions out to production.

Thanks, Free Geek!

PAGIS Presentation Yesterday

Posted on by
Reply

The presentation yesterday went great. At least, we thought so. Giving presentations is now fun. I used to be really nervous talking in front of even a few people. Not so much any more (although this hasn’t been extensively tested on a wide variety of audiences).

In school I would just not do oral presentations. I think some of my teachers must have been sympathetic, but I kind of wish they hadn’t been.

I mentioned that I thought someone from the Oregonian would be there yesterday, but as far as I know, there wasn’t. That’s just as well, because we don’t want quite that much publicity just yet.

The presentation can be viewed online here.

Older presentations can be accessed here.

You may need to adjust the text size in your browser to see everything.

Portland Data Update

Posted on by
1

A couple days ago we updated the Portland region data. This was like a bug-fix release for software–no major changes were made, but a lot of little things here and there were improved and/or fixed.

Related to this (but still quite separate), we improved (hopefully!) the way routes are found. In particular, we made big changes for when the safer option is selected.

Please let us know what you think.

Foray Into Pylons (Python Web Framework)

Posted on by
1

For a while now I’ve been planning to start using one of the many available Python Web frameworks in the development of the trip planner. Over the course of the project we have built an ad hoc, application-specific, CGI framework (of sorts). Using a “real” framework will allow us to stop refining those wheels we reinvented and instead focus more on the specifics of our application.

Deciding which framework to use has been tough. In the past year or so it seems Web frameworks have become all the rage, with several interesting Python projects and all the Ruby on Rails hype. I had initially narrowed it down to two choices, having researched just about all the competitors: TurboGears and Django. I don’t remember Pylons in that first round of research, but maybe I just missed it. Once I noticed it, it was added to the list too.

So, the idea had been floating around for a while and I had created several test applications with various frameworks but still hadn’t gotten around to making a definite decision and switching over. Recently, I listened to a podcast interview of Guido van Rossum[1]. In it he mentioned that Django is his favorite Web framework. He didn’t say much about why except that he likes the way the project is being run. (“They really get open source.”)

Having heard that and having also narrowed the field to just Pylons versus Django, I decided, “OK, well I’m going to go ahead with Django then.” I ordered up a new Ubuntu VPS[2] to host the new version of the trip planner on and got to work. It was exciting to finally make a decision and move forward.

Getting started was easy and I really liked the admin interface, but about five minutes into it, I realized Django wasn’t going to work for me. There was only one reason: the tightly coupled ORM. Sure, I could just not use that part and use a different ORM, but somehow that seems “impure.” Maybe that seems silly, but remember that I had only decided to use Django on, essentially, a whim.

The reason I don’t want to use the built-in ORM is, I want the Web interface to be orthogonal to the core functionality. I want them completely separate, as they are now in the CGI version. I want to keep database access in the back end and be able to create other interfaces as necessary. Now, I suppose I could just use the Django ORM in the back end (?), but there’s another ORM you may have heard of, SQLAlchemy, that I was/am interested in trying out.

This led me back to Pylons and thinking about some of its good points: it’s WSGI from the start, Routes seems like a really cool way to map URLs, and the (Myghty) templates use Python.

Thinking about it, Pylons seems more like the “natural” choice for this project and its architecture. On the other hand, I think Django would be great for projects that don’t need such separation between the front and back ends (or where there is no back end as such).

I’ve just gotten started with the CGI=>Pylons conversion and so far it’s been a pleasure. I’ll report in more detail once I’ve gotten further into it. One “gotcha” that came up, that I don’t think this is specific to Pylons, was creating a controller with the same name as the top-level package. Don’t do something like this:

paster create –template=pylons tripplanner
paster controller tripplanner

Python will be unable to import the controller.

[1] The creator of Python (which I mention for non-technical readers)
[2] vpslink.com (seems good so far)

WordPress Upgrade and K2

Posted on by
Reply

Today we upgraded to WordPress 2.0.4. It’s a security release. Upgrading is easy: just copy the new directory over the old (being careful to save any customizations first, although if you created new directories for your custom themes, you won’t have to do anything).

We also moved from the old default theme, Kubrick, to a new version of it called K2. This version makes a lot of things easier, and it looks better out of the box. One day we might customize it.

Article in the Portland Tribune

Posted on by
1

The article mentioned in this post has been published. It feels a little odd reading about ourselves, but I think it’s a good article overall. Thanks to the Portland Tribune, the writer, and the photographer.

There is one factual error in the article. It states that “…the Portland application currently is restricted to the city limits” and will cover the tri-county area and Clark County (Vancouver) in the future. Washington, Multnomah and Clackamas counties are all already covered, and we are planning to add Clark County in the near future.