The Other Don Kelly

I downloaded the route data for our upcoming trip from Adventure Cyclist as a GPX file, and I also got the Adventure Cycling route app and downloaded the trip there as well. The trip data consists of the route itself (as a path) and the locations (points) of recommended places for food, lodging, bike repairs, and so on; the data is the distillation of their collected wisdom and experience for any given ride. It’s been a goldmine of information for planning our trip, and knowing that it’s based on the knowledge and experience of other travelers makes me a bit more comfortable relying on it.

The GPX is what I got first, and I plan to put the relevant parts of it on my Garmin for the trip, but I opened it in QGIS first because of course I did…

There were six GPX files representing the bike routes as GPX tracks — the main route, a spur to Banff, and a gravel-bike alternate near Fernie, one file for each route in each direction — and one other file with all the services as GPX waypoints. The tracks didn’t contain much information, though the trackpoints themselves did have elevations; the meat of the data was in the service waypoints, and it was interesting to see what information Adventure Cyclist put in for each feature, and how they fit it within the confines of the GPX format.

I don’t usually use GPX except for when I move things to and from my Garmin, so I don’t know too much about it but my impression is that it is highly structured, and, unless you use “extensions,” which not every application will honor or display, it’s a bit rigid in what it can hold — my data has almost always been square pegs, and GPX is all about round holes…

What Adventure Cyclist did was to stuff a lot of the information into the “name” field using initials and abbreviations (“R, CS, M” for restaurant, convenience store and motel, for instance) along with the name, and put the telephone number (along with some travel directions; these were the only contact info given) into both the “comment” and the “description” fields, possibly because different GPS software would look at different fields.

I took this data and massaged it for my own purposes. I also got the app and downloaded the route there. It had the same data, in a very readable and actually beautiful form; it also looked like they maybe used the same GPX data, maybe in another file format but the same structure, and massaged it on the fly. Very interesting…

This got me thinking about my trail amenities map:

• Do I have too much contact information, or not enough? (Answer: my contact information is just right.)

• Am I presenting the information well, especially for use on a phone? (Answer: not really.)

• How should I represent my amenities data, especially for places that have multiple amenities — hotel with restaurant, convenience store with bathroom? (Answer: this will require a whole lot of rework, but I think I should show multiple amenities as multiple symbols in a popup.)

So I am now rethinking my own map based on what I liked about the Adventure Cycling map, but in the meantime I compromised and added some information-massage code of my own, to turn my phone information a clickable link: click on the number (on your phone) and your phone will make the call.

It’s a start.

I make online maps of Shawnee’s Road Scholar bike rides, to help the sag-wagon drivers find things like drop-off and pick-up locations, lunch spots, and other places where they’d meet up with the riders. (I did this after a driver searched for directions to the “D&L trailhead in White Haven,” but there were two trailheads in the town and he went to the wrong one. No biggie, the trailheads were less than a mile apart, but this kind of ambiguity looked like a problem that needed solving.) My first iteration was just a list of all locations in Google Maps’ “My Places,” but now I have a separate, full-blown map (again in Google Maps) for each ride, showing the route itself as well as marked points for the drop-off, pick-up, lunch, and all practical access locations, each with appropriate symbology.

Here is an example for the Lehigh Towpath ride:

(You can view the legend by clicking the button to the left of my name at the top of that map, or view the bigger map by clicking the square at top right.)

I use Google Maps because that’s what the drivers use on their phones for navigation, and this makes it easy for them to navigate to any of my marked points without having to leave the Google Maps universe. To make the maps available, I generate QR codes for each map link; the driver can get the map and open it almost effortlessly with their phone’s barcode scanner. They don’t always use it — they weren’t as dazzled by my geeky brilliance as I was hoping — but hey I was impressed.

The map data comes from a QGIS project I keep for all of the rides we do, or might do, as part of Shawnee’s Road Scholar bike program. My workflow is: select the route for the chosen ride and add it to a KML file, then select the relevant points of interest and append them as another layer in that file. (KML is “Keyhole Markup Language,” Google’s XML-based geographic data format.) I then import the KML file into Google Maps, and I’ve got my route and points on their own new map. A little bit of styling and I’m done — easy peasy! It takes about 5-10 minutes for each map. (The QR codes are generated separately using Jasper Reports.)

And Here My Troubles Began

I have these maps made for all the rides we’ve done, but I have a whole bunch of other rides, either unused or not fully developed, which don’t have their own maps yet, so there are more maps to do. Making each individual map is pretty easy, but I don’t always get the symbology (or the data associated with each feature) exactly the same each time, and I think that automating the task will help with consistency; also, if I start creating a lot more potential rides, the automated map-making could be a potential time saver.

To automate my workflow, I decided to break it down into three basic tasks:

1. extract the data I’d need from the database,
2. use that data to generate a KML file, and
3. import the KML file into Google Maps. (OK, this last step is still pretty manual…)

This is the project I took with me to Colorado, working on it whenever we had an hour or two of downtime — something I can get away with when I vacation with a bunch of readers…

The first step was pretty straightforward, though the SQL (especially for the points of interest) grew into a behemoth, with multiple CTE’s and a whole lot of joins to get things the way I wanted — setting it up just so took a bit of work, but like I said, the task was straightforward.

The next task was more convoluted, and involved a learning curve — I decided to use Python to generate the KML, and this meant using several new packages: geopandas, which makes working with data a bit easier, and simplekml, which, well, does the KML part. Geopandas was pretty easy to use, but simplekml was quirky, maybe a bit too simple, and in the end I actually had to modify the source code to make it work right.

But, in the end I got my KML file (technically, a KMZ file, which included the icons I’d need for styling), and I was able to import it to both Google Earth and Google Maps — the only problem was that Google Maps ignored all my styling! (It worked fine in Google Earth.)

So, somewhere between steps 2 and 3 there was a glitch, a mismatch between what “styles ” meant for KML, and what Google Maps actually used. I spent a good part of the last few days looking at different versions of KML and KMZ files, seeing what worked, or didn’t work, with Google Maps, and I think I have an idea of what’s going on:

Google Maps — or at least, the “My Maps” part of Google Maps — ignores most of the style information, like “use this icon and color,” and seems to code the style information, what they actually use, directly within the style ID’s: a style ID of “icon-1567-9C27B0-normal” would mean “use specific icon #1567, with color #9C27B0 for the normal (un-highlighted) icon.”

So OK, first mystery solved, but what does that get me? I suppose I can use Google’s hard-coded style ID’s, but simplekml does not allow you to specify ID’s or change them, so it looks like that package isn’t as useful as it needs to be, and other Python KML packages seem… not simple.

My next steps? My original plan was to code the entire task in SQL, and I may fall back on that, or I may do something using PHP. I might even take some text-based approach (like I did with geojson a few years ago), since KML is at bottom just a text file. But the truth is, I’m really just going to put this away for a while and let it marinate.