• Tag Archives PostGIS

  • Next Steps for The Native Paths

    Posted on by Don

    So I have about a dozen left of the native paths to add to my database, in this first pass through Indian Paths of Pennsylvania (that book I’m following/analyzing/whatever). This is the pass where I go through the book from beginning to end, adding the basic info for each path to the database, adding the info about the start points and destinations, and generating the routes described in each chapter’s “For the Motorist” section.

    There are a few big pieces left to this project, which I think I can do all together in a second pass through the book:

    1. I need to document the relationships between the paths, as described for each path in the book (which is what required me to go through the book one first time: to get all the paths documented, before trying to map the relationships).
    2. I need to generate the actual footpath routes. This will probably be the most difficult and labor intensive task in the whole project, and I expect it will likely involve digitizing all the (low quality) maps in the book; it may also require trying to find primary sources, old deeds and land grants etc, and even after all that I expect I’ll have to live with a great deal of ambiguity in the routes.
    3. I’m not sure if I want to do this yet, but as I go through the book I may document any points of interest (landmarks, native towns that aren’t trail endpoints) that I haven’t already included.

    I’ve been thinking about the first part for a while, and have set up a separate bridge table in the database to capture these relationships; the table is set up with links to a subject path (the one that’s doing the referring) and an object path (the one getting referenced), and a link to another table with the list of possible relationships between them: intersections, alternate routes and spurs; aliases and alternate names; concurrencies (ie where the path shares some section of trail with another path); and the ever-popular “for more information see also.” I can add more relationships as I see the need.

    For the second task, I think I’ll want to use the maps in the book, even if it’s just to trace over. That means scanning the maps in some way without damaging the book (I may just photograph them with my phone), then georeferencing them and saving the result somewhere. I suspect I’ll end up with a pretty big set of raster data, and now need to consider how to do to organize it. Rasters are not something I have much experience with, so there will likely be a learning curve involved — I think I may put them in the database in some way.

    In terms of original research, my plan at the start was to use Indian Paths of Pennsylvania as my sole source — my project would be the book translated into GIS form — but I’ve already used other sources (e.g., Wikipedia, town websites) to flesh out histories and descriptions, and I think I’m seeing the book now as a condensation of other info, even if it’s just the author’s research files; the info in the book may have been “condensed,” oversimplified, to the point of vagueness, more exact versions of the trail descriptions exist somewhere. I really don’t want to get into actual archival research for this though, and it may just end up that if I dig really deep, I’ll only find that all the primary information is pretty vague too…

    Finally, that third task has me a bit stuck: I’d originally planned to only record the endpoints of the paths (as given in the book), and even called my points table “termini.” Now I’m looking to enter things like landmarks, known trail junctions — there are several places called “the parting of the ways” — towns that aren’t actually endpoints, and all sorts of other points of interest. I painted myself into a corner with that “termini” name, and even if it wouldn’t be too much of a stretch to stick these other points in the termini table, I may add a separate “points of interest” table, or at least add a column to the termini table to designate non-endpoints. (Maybe I’m overthinking this, I could easily find the endpoints and non-endpoints within a mixed table, just by using simple searches.)

    This kind of gets to what I want my native paths data to eventually look like. Many of these landmarks and points of interest are likely to be nodes in a trail network (just like the endpoints), so I am back to thinking they should all be part of the same table. I also expect that I’ll have trail segments from node to node, and my final paths will be lists of trail segments from start point to end point, so my final product will not look much like what I’m building now.

    Well, I still have some time to think about it.


    📂This entry was posted in tech talk 📎and tagged

  • Indian Paths Update

    Posted on by Don

    I’m still cruising along on this project: I’ve got just over 110 paths in the database (of maybe 150 total), about 130 towns or other path endpoints, and 92 motorway routes. I have added no actual paths yet, but the motor routes are starting to look like a real network.

    My current plan is to parse the book three times: once (this time around) to capture the paths, path endpoints, and motor routes; once (the final, and probably most difficult, round) to try and develop the original foot paths; and in between these rounds I will go through the paths/chapters and try to capture all the cross-references between them.

    I noticed early on that there were a lot of things like “this is an extension of that other path,” “so-and-so path also goes by this name,” “this path intersects with these others,” and such like throughout the text; the path descriptions are festooned with these kinds of cross-references.

    (I also finally picked up on the fact that paths without a path/chapter number are not actually part of the previous chapter, but are basically “chapterless,” just the next path name in alphabetical order. They act sort of as placeholders, the alternate names of other, more fully fleshed-out paths — that is, more cross-references.)

    I want to hold on to all this cross-reference information in my database, so I set up a bridge table to work something like a resource description framework, with the referring path as the subject, the referenced path as the object, and for the predicate I would use a description of the relationship type, such as “[subject path] is a continuation of [object path],” “[object path’s name] is an alternate name for [subject path],” “for more info see [object],” and so on. I now have all of this set up and ready to go, but before filling it in with information I want to have all the paths already in the database. Soon…

    Meanwhile, the details, of each path or town I add, have all been real eye-openers. I often do a little internet research on each town, or village, or Native name I come across, and each bit of info, each piece of the puzzle is another portal into that era.


    📂This entry was posted in looking back tech talk 📎and tagged

  • Milestone

    Posted on by Don

    So I’ve been messing with that Lehigh Valley bike commuter routing program again, and I have made some important strides:

    • I found a way to update the recommend routes easy/advanced, etc, by maintaining separate tables of these routes as linestrings (which I can add and subtract, draw and redraw), then updating the relevant field in the main table using a spatial join. The update process is now automated simply by running SQL files, one for each type of route.
    • I sat down with the workflow for updating the main map table, and managed to automate much of it — everything but the SAGA tasks, though I think I can automate them too, eventually. I also managed to streamline one of the more time-consuming tasks: Generating the ascent/descent tables used to take upwards of 12 hours when I first did it (using Python within QGIS), and my next iteration (using PostGIS) took about 20 minutes, but my latest method got it down to about 57 seconds. Fifty-seven seconds! All of these are now also stored as SQL files or functions, so they are available almost at the push of a button. (My goal is a shell script putting all of this together.)

    These were both pretty big deals, since they were the only things keeping the project from being truly functional. Before this, keeping the database up-to-date was like pulling teeth. Unfortunately, I decided to add some front-end functionality, testing to see if selected points are within the Lehigh Valley, and that’s been a bit of a struggle, but if I can find a host for the project I think I can go live really soon.

     


    📂This entry was posted in tech talk 📎and tagged