• Tag Archives GIS
  • Anything to do with Geographical Information Systems or mapping.

  • New Maps Marker Plugin Test

    Here’s another map plugin, called “Maps Marker.” This one is also based on Leaflet, but seems to have many more features, at least at first. Here’s a simple map:

    KML-LogoFullscreen-LogoGeoRSS-Logo
    Hotels

    loading map - please wait...

    Hotel Bethlehem: 40.620044, -75.382369
    Sayre Mansion: 40.611965, -75.384450

    This may take some work to bring it up to speed: you have to load your own marker icons etc, and many features (e.g., GPX tracks) are unavailable, unless you get the Pro version — at €249 for the license, no thanks! Meantime, the actual markers didn’t even show up on the map (though they did show up on the post preview), let alone the pop-ups — again, this may be a problem with my theme, though explicitly creating a map for one specific marker does show up (sans pop-up or tooltip):

    Sayre Mansion

    loading map - please wait...

    Sayre Mansion 40.611965, -75.384450 The Sayre Mansion is a bed and breakfast on the south side of the river.

    It looked for a second like we were getting close, but still no cigar.


  • New Leaflet Plugin Test

    I’ve decided to make my own maps here for posting about rides, and maybe for some other mapping tasks I might want to do, so I’m experimenting with map plugins. The first one is something called “Leaflet Map,” let’s see how this looks:

    Hmmm, seems OK, but there aren’t all that many options straight out of the box — I especially would like to be able to center the map, for one thing, and the map design (on the back end) seems pretty text based. Default tile server is standard OpenStreetMap, but there is an option for Mapbox (with a place for your API key), as well as other map tile providers — though I don’t see how to set the API key for anything other than Mapbox. This could be a problem for, say, ThunderForest tiles, which are my favorite map tiles and also require a key.

    Meanwhile, try clicking on the marker: a pop-up appears, but the pop-up is apparently only one letter in width, and its background is transparent, and I don’t see any easy way to change these. (These may be due to issues with my WP theme, but I like my theme and I don’t want to change it.) Like I said: Hmmm…

    If I keep using this map plugin, I may have to build a custom container for it, div tags and custom css, with a place for titles, etc. I have other plugins to try before it comes to that.


  • Data Cleanup Automation Fun

    I’m still playing with LANTA’s bus routes data, girding myself for — eventually — adding the bus routes into OpenStreetMap, but I recently decided to rebuild my data, basically starting over from scratch with the PDF’s I got from the LANTA website. My current workflow is:

    PDF  —> CSV —> cleaned-up CSV —> GeoJSON —> cleaned-up GeoJSON —> (eventually) a PostGIS database

    The conversion from PDF to CSV was automatic, using a Java program I found online, and the CSV cleanup — fixing things like transposed latitude/longitude, missing minus signs etc — was done manually (using a text editor and LibreOffice Calc), which was relatively uneventful but  laborious — there are 68 individual bus routes, each with its own file.

    Things got even more laborious with the next two tasks. My first go-around with the conversion to GeoJSON was done manually within QGIS: load each of the CSV files, individually filling out the required parameters for each one. I wasn’t looking forward to converting the files individually, so I wrote a Python script to save all my route layers in GeoJSON format. (Just as an aside, I have to say I really like GeoJSON as a vector file format: I find it much easier to work with than the dated, unwieldy Shapefile standard; it’s also easier to open and work with in the JOSM editor. All my new data, if it’s not going into the database, is getting stored as GeoJSON files.)

    The “GeoJSON cleanup” is where I massage the data into the forms I want: some of the table columns are unnecessary, some need to be renamed, there are a few extra columns to add (and populate), and finally I wanted to convert the LANTA bus stop names format (in  Robbie-the-Robot-style ALL CAPS) to something a little easier o the eyes. Doing this manually would have been beyond laborious, so I wrote another Python script to massage the route files. This turned out to be more of a learning experience — as in, multiple versions of the program failed spectacularly until I got it right — and probably took longer than the brute force, manual changes approach would have, but at least it wasn’t laborious…

    I’m still not really sure why this version worked when others did not, but here is my code:

    # script to run through all visible layers,
    # adding/deleting/renaming fields as required
    # to convert from LANTA bus data to something
    # more like OpenStreetMap bus stop attributes
    # it also properly capitalizes feature names

    from PyQt4.QtCore import QVariant
    import re

    # some functions and regular expressions for string manipulation
    def match_lower( matchobj ):
    return matchobj.group().lower()
    def match_upper ( matchobj ):
    return matchobj.group().upper()
    reg = re.compile( 'Lati|Longi|Time|At Street|On Street|Direct|Placem' )
    reg_ns = re.compile( r'\(ns|fs|mid|off\)|\bncc\b|\bhs\b', re.IGNORECASE )
    reg_nth = re.compile( r'[1-9]?[0-9][a-z]{,2}', re.IGNORECASE )
    reg_leftParen = re.compile( r'([^\s])(\(\w*\))' )
    reg_rightParen = re.compile( r'(\))([^\s])' )
    reg_space = re.compile( r'\s+' )

    for layer in iface.mapCanvas().layers():

    # reset these variables for each new layer processed

    myLayerName = layer.name()
    highwayExists = False
    networkExists = False
    operatorExists = False
    publicExists = False
    busExists = False
    routeExists = False
    delList=[]

    layer.startEditing()
    pr = layer.dataProvider()

    # change the names of some fields
    fields = pr.fields()
    count = 0
    for field in fields:
    fieldName = field.name()
    print "test for changing field name " + fieldName + " count ", count
    if ( fieldName == 'Public Information Name' ):
    print 'changing to name'
    layer.renameAttribute( count, 'name' )
    if ( fieldName == 'Stop Number' ):
    print 'changing to ref'
    layer.renameAttribute( count, 'ref' )
    if ( fieldName == 'Stop Order' ):
    print 'changing to stop_order'
    layer.renameAttribute( count, 'stop_order' )
    count += 1
    layer.updateFields()

    layer.commitChanges()
    layer.reload()
    layer.startEditing()
    pr = layer.dataProvider()

    # delete some fields
    fields = pr.fields()
    count = 0
    for field in fields:
    fieldName = field.name()
    print "test for deleting fields " + fieldName + " count ", count
    m = reg.match( fieldName )
    if m:
    print fieldName
    delList.append( count )
    print delList
    count += 1
    pr.deleteAttributes(delList)
    layer.updateFields()

    layer.commitChanges()
    layer.reload()
    layer.startEditing()
    pr = layer.dataProvider()

    # add some fields, checking if they don't already exist
    count = 0
    fields = pr.fields()
    for field in fields:
    fieldName = field.name()
    print "test for adding fields " + fieldName + " count ", count
    if( fieldName == 'highway' ):
    highwayExists = True
    print 'highway', count
    if( fieldName == 'network' ):
    networkExists = True
    print 'network', count
    if(fieldName == 'public_transport' ):
    publicExists = True
    if(fieldName == 'bus' ):
    busExists = True
    if ( fieldName == 'operator' ):
    operatorExists = True
    if( field.name() == 'route' ):
    routeExists = True
    count += 1
    if( not highwayExists ):
    print "adding highway"
    pr.addAttributes( [ QgsField("highway", QVariant.String) ] )
    layer.updateFields()
    if( not networkExists ):
    print "adding network"
    pr.addAttributes( [ QgsField("network", QVariant.String) ] )
    if( not operatorExists ):
    print "adding operator"
    pr.addAttributes( [ QgsField("operator", QVariant.String) ] )
    if( not publicExists ):
    print "adding public_transportation"
    pr.addAttributes( [ QgsField("public_transport", QVariant.String) ] )
    if( not busExists ):
    print "adding bus"
    pr.addAttributes( [ QgsField("bus", QVariant.String) ] )
    if not routeExists:
    print "adding route"
    pr.addAttributes( [ QgsField('route', QVariant.String) ] )
    layer.updateFields()

    # add attributes to new fields for all features
    for feature in layer.getFeatures():
    feature['highway'] = "bus_stop"
    feature['network'] = 'LANTA'
    feature['operator'] = 'Lehigh and Northampton Transportation Authority'
    feature['public_transport'] = 'platform'
    feature['bus'] = 'yes'
    feature['route'] = myLayerName
    layer.updateFeature(feature)
    layer.updateFields()

    # clean up text in name field
    fieldName = 'name'
    for feature in layer.getFeatures():
    myString = feature[fieldName]
    myString = myString.title()
    myString = reg_ns.sub( match_upper, myString )
    myString = reg_nth.sub( match_lower, myString )
    myString = reg_space.sub( ' ', myString )
    myString = reg_leftParen.sub( r'\1 \2 ', myString )
    mystring = reg_rightParen.sub( r'\1 \2', myString )
    print myString
    feature[fieldName] = myString
    layer.updateFeature(feature)

    layer.commitChanges()
    layer.reload()

    Once I got this up and running, I realized that I wanted t do some more preliminary cleanup on my spreadsheets, so I was back to square one. I couldn’t really find out how to do a bulk load of my CSV files into QGIS, and I realized that QGIS was just using ogr2ogr under the hood, so I decided to do the bulk converting, CSV to GeoJSON, with a shell script that calls ogr2ogr. Yet another learning curve later, and it works great. More code:

    for i in *.csv
    do
    echo $i
    tail -n+2 $i | ogr2ogr -nln ${i%.csv} -f "GeoJSON" ${i%csv}geojson \
    CSV:/vsistdin/ -oo X_POSSIBLE_NAMES=Lon* -oo Y_POSSIBLE_NAMES=Lat* -oo KEEP_GEOM_COLUMNS=NO
    ogrinfo ${i%csv}geojson
    done

    It struck me then, that all my data was really text, and so working with it in a more unix-ey fashion, with shell scripting and text manipulation programs (sed, awk) to do the conversions directly from the CSV files, was probably my better strategy. Oh well, I did the first part of it with bash at least, and the Python script works well enough.

    UPDATE:

    I did it anyway, using awk to add/subtract/rename/Capitalize the CSV data before running it through ogr2ogr. The code (below) is about 35 lines (as opposed to 150 for the Python script, which only does part of the job anyway) and runs really fast:

    #! /bin/bash
    for i in *.csv
    do
    echo Converting $i to GeoJSON
    cat $i | awk -F "," -v rtname=${i%.csv} 'BEGIN {
    }
    $2 != "" && /Stop/ {
    print "ref,Latitude,Longitude,name,stop_order,highway,public_transport,bus,network,operator,route"
    }
    $2 != "" && /^[0-9]{4,4}/ {
    string = tolower($8)
    n=split(string,a," ")
    string=toupper(substr(a[1],1,1)) substr(a[1],2)
    for(i=2;i<=n;i++) {
    string = string " " toupper(substr(a[i],1,1)) substr(a[i],2)
    }
    t_str = "\(ns\)|\(fs\)|\(mid\)|\b[nN]cc\b|\b[Hh]s\b"
    if ( match( string, t_str, match_array ) ) {
    the_match = toupper(match_array[0])
    gsub(t_str, the_match, string)
    }
    gsub(/ *\(/, " (", string)
    $8 = string
    print $1 "," $2 "," $3 "," $8 "," $9 ",bus_stop,platform,yes,LANTA,Lehigh and Northampton Transportation Authority," rtname
    }' > test1.csv
    cat test1.csv
    ogr2ogr -nln ${i%.csv} -f "GeoJSON" ${i%csv}geojson test1.csv -oo KEEP_GEOM_COLUMNS=NO
    ogrinfo ${i%csv}geojson
    rm test1.csv
    done


  • This Week Today

    Stopping by again…

    Mapping: I had, and still have, a few technical issues to deal with, but the full Lehigh Valley database is now in PostGIS, along with elevation data — bogus elevation data, that’s one of my technical issues — and the demo map can now route with the new database. But it’s got the slows, it’s got the slooowws… With about 3200 road segments in the “toy database,” it could route in about 1-2 seconds, but the full-map version took about 6 seconds per routing task — and there may be multiple routing tasks in each route, from start point, to via point and then through subsequent via points, and finally to the endpoint. Unacceptable!

    I did some searches online, and sure enough there are a lot of people complaining about pgRouting performance and looking to speed it up. The general consensus: there are a few things you can do, including tune your database, but the actual bottlenecks are the pgRouting algorithms. Some suggested using osm2po, another program that converts OpenStreetMap data for databases but can  also do routing: tried it and it’s blindingly fast – d’oh! (Unfortunately, I didn’t see much there in the way of customized, dynamic cost functions, so I can’t see how to turn it into the the answer I’m looking for.) I tried a bunch f the Postgres/PostGIS performance-tuning tips anyway, and they did seem to help a little.

    I eventually came across one potential solution: route only on a subset of the roads in the database, using a bounding box. For each pair of points to route between, I find the smallest rectangle that contains both, then expand it by 2000 meters in every direction (like a buffer zone); this is my bounding box, and the routing search is limited to the roads that touch or fall within that box. This seemed to do the trick: my routing times are back down to about 1-2 seconds.

    Except near — wait for it — those confounded bridges. The valley is broken up by the Lehigh river, with occasional bridges, and if there are no bridges within the bounding box for a route that needs to cross the river, no route will be found. Meanwhile, when routing points are on a diagonal, the bounding boxes are fairly big, but routing points that run mainly east-west or north-south produce long, skinny bounding boxes. I found a few “dead zones” where routes couldn’t be found, especially east-west ones north of Northampton, routes with skinny bounding boxes where the bridges are a little sparser. My original bounding boxes were expanded by a buffer that was only 1000 meters; I went to 2000 meters in an attempt to alleviate the bridge problem. This didn’t solve it entirely, but it did help, and there was no real performance hit going from 1000 to 2000 meters. I’ll probably look at distances between bridges, and revise my buffer zone to be just bigger than say, half that distance.

    Reading: I picked up Don DeLillo’s Underworld again, intending to just read the first part. I love the first chapter but never finished the book because I found the rest boring; now I am engrossed and don’t know what I was thinking back  then.

    Listening: WXPN has been playing “The 70’s, A-Z” this past week, every song they have in their library that was released in the Seventies, played in alphabetical order. We’ve been following along religiously, and it’s been fascinating and fun but they’re only up to “T,” and it gets wearing. Full disclosure: the radio is off right now…

    The only time they weren’t playing the 70’s was for their Friday “Free at Noon” concert at the Word Cafe, which this week featured Russ’s band Cherry. So, we went down to Philly with Ray and Lorraine, where we met Frank and Patricia, and Ben, and Gabby, and we all watched the show and then went out to lunch with Russ at the White Dog Cafe. As always, we spent a few minutes at Penn Books before the ride home. All the talk in Philly, among us and overheard on the street, was about the upcoming snow on Saturday…

    By the way, Saturday was Luminaria Night in Bethlehem, here is a photo of ours:

    candles in bags on sidewalk
    Luminaria Night

    One last thing: here is what I wrote ten years ago.


  • On The Bridge: One Step Forward, Two Steps Back

    Talk about hedonic adaptation! A week or so ago I was sure I was far from ever being able to route on my web map using pgRouting, then a triumphant breakthrough, and now here I am, annoyed that it’s not perfect…

    My first problem is a data issue, and a recurring one in my mapping and routing life: dealing with bridges. Once I got the routing to work, I started to customize it with a separate “get_cost” function, which deals with ascent and descent (other criteria are coming), and that worked fine. Then I noticed that the routes seemed a little off, like they were avoiding what I thought would be the optimal routes, the main one being that it would do a lot to avoid a certain section of Broad Street. That’s when I remembered: there are several bridges on that section, and rather than following the elevation of the bridge’s road surface, my elevation data followed the contours of the ground below it, leading to large ascents and descents along that section.

    To solve the immediate problem, I changed the ascent and descent to be zero for the section containing the bridge — close enough to the truth, for that short a span on a mostly flat road. That made the routes in the vicinity more sensible, but what to do about other bridges?

    I think I have three options: I can either find the actual elevation data for the top of the road surface (using a “digital surface model” rather than “digital elevation model” and probably using LIDAR rather than satellite radar data), or I can assume that the bridge has a mostly constant slope, and calculate the slope from elevations where it attaches to the ground, or I can save myself a lot of work and just say “they’re flat, or flat enough to make no difference,” and make all ascents and descents be zero for bridges. I am still thinking about this…

    The second problem is a little harder to figure out, since it involves the PostGIS routing function I got off the Internet: when the beginning and end points of the route are on the same segment of road (ie there are no intersections between them), the function fails. I don’t know enough about Postgres functions to be able to solve this, so I may have to either live with it for a while, and contact the person who wrote the function for some help.


  • Excelsior!

    I thought this would take so much longer… I have a web map designed and working, and I was able to add generic routing functionality to it (via the Leaflet Routing Machine plug-in). Here’s a screenshot:

     

    A screenshot from my web map showing a path around town, using the Leaflet Routing Machine and my PostGIS/pgRouting database.

    The actual routing engine on the server side is done with PostGIS/PgRouting, accessed through a PHP script — Leaflet Routing Machine does not normally work with PostGIS,  but I found another plug-in (called lrm-pgrouting), which included a PostGIS function and some Javascript, to add this functonality. Unfortunately, the unmodified lrm-pgrouting plug-in requires something called Geoserver (rather than just a plain old PHP script) to talk to PostGIS, but it was easy enough to modify the plug-in to use PHP, and writing the script itself was the easiest part of the whole thing.

    Once I got routing up and running, I came across a few problems, mostly with how the directions are displayed: the display only showed street names and distances — that is, no turn information — and, for wherever the route continues straight on the same road through an intersection, there is an unnecessary instruction to “go straight.” Both of these were due to errors in lrm-pgrouting which I managed to fix to my liking.

    So, success! But this is using a generic routing function, rather than the “climbing vs busy road vs recommended commuter routes” function I ultimately want to use, so there’s some room for improvement. I also think I may want to abandon the Leaflet Routing Machine and do the input and display on my own. Also also, all this stuff currently resides only on my laptop; I have to find a host that will let me run a PostGIS database (among other things). I still have some work to do.

     


  • Go, But At What Cost?

    Listening: “Golden Brown” by the Stranglers

    Well, so much for my previous experiment — don’t compartmentalize, just write — the real task is to actually take the time to write something. I haven’t been motivated lately  to do any writing, but in my defense there have been a lot of things to write about, which might have caused a bit of vapor-lock. Among other things, I’ve still been volunteering at the Canal Museum and at CAT, we did another overnight trip up to Jim Thorpe, and I’ve been exploring mapping and routing again.

    Listening: “Colossal” by Wolfmother

    Listening: “Beautiful Red Dress” by Laurie Anderson

    ON THE BIKE:

    We did another trip up the towpath: Anne and me, Sarah A and Dianna H. We rode to Jim Thorpe, lunching in town and camping overnight at the lake. Breakfast in town, then Anne continued north on her own from there while Sarah and I did the return trip (Dianna met her husband in town and got a ride home). Anne’s destination was Watkins Glen, and I caught up with her by car there a few days later. I brought my road bike, but we didn’t do much cycling, just some hiking at the Glen, then a trip to the Museum of Glass in Corning on the way home.

    Listening: “Help Me Mary” by Liz Phair

    I kind of got the mountain biking itch again: I did a ride on some seriously “old school” trails, with Greg H up in the Poconos near his cabin, probably my favorite ride of the year so far (except for a persistent creaking out of my pivots). Got the pivots fixed, rode Nox on a weekday with Anne, and did Deer Path/Pine Tar in Jim Thorpe Sunday. I’ll be doing a towpath ride later today.

    In between these things, I also took the Cycling Savvy course. Really fun, and though it covered a lot of the same ground as Road 1/LCI, I came away with more than a few choice new insights. I also rode across the Hill-to-Hill Bridge with my sister-in-law, which should have been a crazy idea, but by the time we did it (as part of the course), it was easy enough to be a bit anticlimactic.

    Listening:“Drumming Song” by Florence + The Machine

    We also managed to do some riding as part of Lehigh’s Car-free Day.

    Listening: “Make You Mine” by Heather Nova

    All this music just lets you know how slowly I write, and also how many in my “Favorites” playlist are female vocalists…

    READING

    I re-read The Mathematician’s Shiva recently, as well as all the “Expanse” books (which didn’t hold up to a re-read as well as I would have liked), and China Mieville’s Iron Council (ditto), the most recent new book was Walkable City by city planner and walkability expert/advocate Jeff Speck. Interestingly, he once was commissioned to do a study of Bethlehem, and gave a talk at Lehigh about his findings. (The town skipped over a bunch of his advice, but they did incorporate at least some of it, some parts more slowly than others.) It was fun (at first) to see him name-drop Bethlehem, and CAT, likely referring back to his study, but it became annoying after a while since it was mostly examples of what we were doing wrong…

    Case in point: I had just finished the “Cycling Savvy” course when I got to Speck’s critique of “Vehicular Cycling,” where bicyclists are trained to bike (on the roads) as drivers of vehicles — in other words, “Road 1” and “Cycling Savvy.” His contention, and there is some merit to it, is that while this may help an individual graduate be safer, it makes cycling grim and scary, a turn-off, thus reducing the number of actual cyclists on the road — and since the biggest driver of cycling safety is not cyclist skill (or wearing a helmet or whatnot), but the number of cyclists on the road, the vehicular cycling approach actually reduces general cycling safety. Oh well, he has a point, but I still liked biking over that bridge.

    Listening: “Old World” by The Modern Lovers

    FUN WITH POSTGIS

    I’ve been playing with a new project recently: building a web map for cycle commuters in the Valley. In the end it will show the major Lehigh Valley towns, and the locations of the major employers, and recommended routes that a cycle commuter might use to get around; I used Leaflet to get these basics down, but then I thought that what the map really needs is routing, and I thought it would be best to build a custom routing engine using PostGIS and pgRouting.

    Listening: “Wildewoman” by Lucius

    So, I’m back to my routing kick; this will be part 2 but I’ll be abandoning my previous project in favor of the web map.

    Listening: “Twenty-first Century Schizoid Man” by King Crimson

    First step (of many) was to make sure I had pgRouting installed properly along with PostGIS, and they both were, no problem. Next up was to build my road network — for right now I’m working on a smaller area, a part of Bethlehem City. I got the road data from OpenStreetMap and used the osm2pgrouting utility to get the roads into the database. So far so good, and the whole process was surprisingly easy.

    Using the routing functions took me a while to work out, but in the end they were also pretty straightforward. PostGIS/pgRouting seem to be easier to use, and easier to do sophisticated things with, than the original QGIS networking utilities.

    Listening: “White Unicorn” by Wolfmother (oh no the same band again!)

    Two things in particular came more easily: dealing with one-way streets, which I ignored in the first project since it seemed more trouble than it was worth (you could always walk your bike) and the actual “cost” of cycling.

    The basic idea behind routing is to find a path through the network that minimizes some function, the total of the “costs” of moving from each individual point to point within the network. The default cost function for my first project was distance (the default, and by far the easiest thing to do), which is a pretty good cost function as far as it goes. But with bicycling, elevation changes could also play a major role, and with pgRouting it’s easy enough to define your own costs.

    Listening: “Funkytown” by Lipps Inc.

    So I decided to consider total ascent, every meter climbed, as part of the cost; I found some studies that cyclists generally take a meter of ascent as equivalent to eight meters of travel — that is, you  might go eight meters longer to avoid another meter of climbing. I also thought that grade would affect that eight meters, and found another online study that multiplied ascent by a factor proportional to some power of grade. I eventually opted to go with a geometric factor, doubling elevation cost every 5% change in slope.

    Listening: “Strangeness and Charm” by Florence + The Machine

    I’ll get into it some other time, but I got QGIS to break up my network for finding ascent, and SAGA to assign elevations from DEM data, then wrote a Python script to extract and calculate the elevation costs. Took some doing, mostly dealing with my own typos, but finally I got the whole thing to work, and it routes beautifully — in QGIS, on my machine.

    Listening: “There There” by Radiohead

    I learned a few interesting things about elevation along the way. A sanity check of my cost results showed some anomalies, especially on 8th Avenue — it turns out my original “elevation data” was raw Space Shuttle radar data, and it picked up the top of the old Martin tower which screwed up nearby elevations. (This difference between ground elevation and radar/lidar elevation readings, the realm of buildings and trees, is — according to the Internet — very important to telecommunications people, who call it “clutter.”)

    So for my second iteration I used actual DEM data  (“digital elevation model,” the elevations at the surface of the Earth, as if it were scraped clear of buildings and vegetation), and that fixed the Martin Tower problem but revealed another one: bridges, my nemesis… I’ll have to figure out how to adjust elevations on bridges so they don’t follow the depressions (creeks, rivers) they jump over.

    Listening: “Furr” by Blitzen Trapper

    Well, that’s it for now…

    “But I still dream of running careless through the snow,

    And through the howling winds that blow

    Across the ancient, distant flow

    To fill our bodies up like water till we know.”

    … now let’s ride!

     


  • Experiments in Routing, Part 2

    This is my second post in a series, where I report back on my results from playing with various ways to use routing, in QGIS and related programs. My immediate task is to identify those cycling amenities that are nearest to access points along the Lehigh Towpath. You can read Part 1 (the introduction) here. In this post, I’ll be using the QGIS built-in Network Analysis Library. Follow along after the break…

    Continue reading  Post ID 1232


  • Experiments in Routing, Part 1

    This is the introductory post for a hopefully four-part series about using QGIS to find the shortest path between two points, not shortest as the crow flies, but following a given network of roads. This is called routing, it’s what’s Google Maps and other mapping software uses, and it relies on graph theory and network analysis to do its job. I’ll talk about the what and the why of this little experiment here; the how (for three different versions of how) will be the subject of subsequent posts.

    UPDATE: Part 2 can be found here.

    The reason I’m looking at all this goes back to my interest in cycling tourism, and my attempts to identify cycling-accessible amenities — convenience stores, restaurants, hotels, that sort of thing — along the Lehigh Towpath. My first attempt (you can find it here) basically looked at a region, within a mile (as the crow flies) of one section of the Lehigh River, and searching within that region for the amenities I was interested in. That was an interesting project in its own right, but, as I said in my earlier post, it didn’t really solve the right problem:  there are many places within a mile, or even a quarter mile of the river, that are not anywhere near accessible from the towpath: they may be on the wrong side of the river, say, or not near a towpath access point. To be considered accessible, the points of interest would need to be within a mile (or whatever arbitrary distance I end up choosing), by road, of an access point on the towpath.

    map of roads near lehigh river
    Data gathered, and almost ready for routing.

    I didn’t really have a plan to make this happen yet, but with or without a specific plan, I figured my first order of business was to get the information I would use. That previous analysis used Google Maps, but I felt that their data was a bit encumbered (in terms of my rights to it), and it seemed that Google didn’t play as well as I’d like with QGIS anyway, so I decided to use the data available through Open Streetmap, for both the road network and the set of amenities. (I already had a collection of the towpath access point locations left over from a previous experiment.) I got those sets of data, and massaged them so that I only had the parts that fell within a mile of the bike paths in the Lehigh Valley. This gave me the data seen to the right, where the aqua lines are the road network, the red lines are bike trails (the towpath, plus the Palmer Bike Path), the red stars are trail access points, and the orange dots are the amenities (restaurants, fast food etc).

    (One note about the road network: You probably can’t see it at this resolution, but I made a point of excluding roads that are not practical/legal/safe for cycling, like US-22, I-78 and a few others. There are also a number of places, like the New Street and Hill-to-Hill Bridges, where roads or the trail are connected via stairways to the bridges above; after our own struggles, a few years ago, with stairs and fully loaded touring bikes at the Ben Franklin Bridge, I decided to also exclude stairways from my network.)

    So that gets us the data, what about the analysis? My first thoughts were to see if I could find all the points on the road network that were a mile away from an access point, then connect the dots to define a region, and then find all the amenities within that region. My second thoughts were that this approach would put me back in the same situation as my first attempt, since I could easily find roads that were not reachable within that region, such as bridges. (Bridges became my nemesis for a while.) I eventually decided that my best strategy would be to find the shortest route between each access point and each amenity, and select from the amenities based on the lengths of the routes I found.

    To perform the actual routing analysis, I have three options:

    In terms of a learning curve, I have some experience with networks in GRASS, and I feel at least a little comfortable with Python (and copy-paste, with scripts I find online), so pgRouting will probably be the most difficult for me to pick up. Meanwhile, the Network Analysis library can use the data I already have, but Open Streetmap deals with road networks in a way that’s not directly compatible with either GRASS or pgRouting — their topological models are different, but that’s an issue for a future post. I would have to either re-import the road network to get it to work with pgRouting, or further process the one I have for GRASS.

    Each one of these approaches will be the subject of its own post. Given that the Python approach is not the hardest, and my data is already in the form I’d need, I am going to try my hand with the Network Analysis library first. Stay tuned for Part 2, whenever…