Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.
I just thought I’d post an update on my new computer: it’s been assembled, it’s been shipped and it’s due to arrive here this Wednesday. I can’t wait!
Meantime, I was looking into what was going wrong with my GeoPackages, and I found out that there are problems with (my now slightly outdated versions of) QGIS and Spatialite that prevent GeoPackage spatial queries from working. So it wasn’t me!
I’ve done a few more Road Scholar gigs this year, and my co-guide and I both feel that the ride choices could be improved, mainly by doing more bike paths and rail-trails, and doing less actual road riding. This would avoid the biggest issues we face (traffic and hills), and maybe allow the rides to be a bit longer and more enjoyable.
Meantime, I’d noticed a tendency, among our van drivers, to use Google Maps to navigate our pick-up, drop-off and other van access points. This is I think a good thing, but it’s led to map searches finding the wrong drop-off point — nearby features rather than the specific location we use. It works well enough that “OK, turn left here and pull into that parking lot” will get us there once we’re close enough, but navigating to an actual position (a given latitude and longitude, for instance) would work much better.
Finally, I thought it would be good to have an official repository somewhere, of the rides: their official routes (I use GPS to navigate on the rides) as well as waypoints, like lunch spots, points of interest along the ride, and those pick-up and drop-off points. Ideally, I would be able to load a ride into my GPS and have all info for the ride at my fingertips.
These all coalesced in my mind into the Great Big Ride Database GIS Project. The project would be made of three parts: storage of rides (official or otherwise) and waypoints into a ride database, transfer of rides/waypoints to and from my GPS, and analysis of the ride data.
I started by keeping “official versions” of our rides on RideWithGPS, and I would download them as GPX files onto my Garmin when I needed them. This would only take care of the route itself, however; I thought that there was also a need to maintain a list of waypoints associated with each route, so I decided to build some kind of database to hold routes and their waypoints.
Since I would like to be able to just hand over the ride information in some file format, my first attempt was to build the database as a GeoPackage file. This actually worked pretty well, when my plan was just to stuff the data into storage. But then, my plans started to morph: I needed to actually analyze the data (with a spatial query) to generate info I needed. The GeoPackage file should have been able to handle this, but I think I must have done something wrong back when I installed the underlying GeoPackage/SpatiaLite libraries, or I was doing something wrong now, but I just couldn’t get any spatial functions to work. After frustrating myself for a while I just moved the database over to PostGIS. My project was changing, but at least it worked.
So at this point, I started looking at the problem of getting the point data to places where I could use it — like onto and off of my Garmin. I collected a bunch of the waypoints as “saved locations” on my GPS, but then I couldn’t find any good way to export or upload them. (The Google tells me that Garmin apparently has some Windows programs that can manage waypoints, but that does me no good.)
I eventually dropped back and punted by writing a Python script. I scrounged around inside my Garmin and found a file called Locations.fit that seemed to be where the saved locations were stored, and used that fitparse library to rummage inside the FIT file, eventually figuring out the (undocumented) structure used to store waypoints. I could now export the waypoints into a QGIS layer, then I managed to realize that I could import the waypoints to my GPS via a GPX file in the same way I could import rides via GPX, and could even combine waypoints with the ride trackpoints in the same file for importing. Major breakthrough! — though the Garmin seemingly ignores all waypoint information (symbology, comment) except the name.
So things are now a bit different than how I first planned it, but I have a system that works. Next up: evaluating potential routes.
The laptop is starting to really show ominous signs of impending demise — screen frequently goes static-ey or blank on startup, wi-fi and bluetooth unexpectedly crap out, and it no longer can wake up from sleep mode — things that started happening a few months ago but are now getting to be almost constant, or at least daily, occurrences.
I’ve had my laptop (a Dell XPS M1530) for 10 years now; it was cutting edge when I bought, it but a decade of advancement has left it so far behind that some software can’t even be updated anymore. But I love this computer even so, and if it weren’t on the verge of crapping out I wouldn’t even think of replacement. It’s lasted probably twice as long as it should though, and owes me nothing at this point, as long as it lasts until the new one shows up…
So anyway, I went online yesterday and bought my new one, a System76 Darter Pro. I wanted another 15.6″ screen, I think that’s the sweet spot for laptops, and I also wanted Linux pre-installed. I was also looking for reasonable battery life, light weight etc, and I also wanted something with a bit of pep, even by current standards. I think I compromised a bit by getting an i5 rather than an i7 chip, but I turned around and upgraded to 16G of RAM and a 500G NVMe solid-state drive, improvements at the likely performance bottlenecks, so I think I’ll get a good bang for my buck. It should arrive in just under two weeks.
I finally made good on the promise I made to Anne, to my brother-in-law Ray — to anyone who would listen, really — to get a new headset for the Iguana, after that bike-pukes-ball-bearings debacle at Mauch Chunk Lake. I ordered the headset (from Saucon Valley Bikes) last week, brought the bike in a few days ago for them to install it, and picked it up this afternoon. I haven’t done a test ride yet, but it feels great.
Once I got it home I took apart the rear grip shifter and lubed it, and replaced the shifter cable and housing while I was at it. This took longer than I expected, since I forgot how to work with grip shifters, but eventually it all came together — I have now resolved two decade-old problem areas.
Right in the nick of time too, because next week I’m back to the Road Scholar tours, and the Iguana is my go-to bike for these rides because it can carry panniers.
Now that I have my new Garmin device, I’ve been thinking of picking up where I left off with that ride database and viewing project, which I haven’t touched in months. So imagine my surprise when I go to open Eclipse, my IDE of choice for just about all things coding, and it’s missing!
I have no idea what happened. Sometime between about March and yesterday it deleted itself from my machine — maybe some other program deleted it, or maybe I did it myself without realizing, but bottom line: it was definitely gone. I checked, and the data/programs I’d been working on were still on my machine (whew!) so all I had to do was re-install the program…
It turned out not to be so simple: the latest versions all require 64-bit systems (my dinosaur of a laptop is 32-bit), and the version provided in my outmoded distribution (Linux Mint 17) was far older than the one I had previously installed. Luckily I found, buried in my archives, the program installer for my previous version. I ran that, which gave me a bare-bones installation, and then I started adding on the other modules I used to use. After that came all my personalized configurations… It took a while to get things back to what I think I remember my old system looked like, but everything seems pretty normal again now.
I’m not ready to start messing with the code just yet, but it’s a big load off my mind to know that my setup is back in place.
I’ve been embedding too many rides lately so I won’t post the data, but I did two rides at Nox this weekend and I learned something about my new Garmin: it beeps when you get air.
Friday night I was out once again with John and Renee, along with Scott & Erin and Heckler Mike, for a “happy hour” ride. We started at the boat launch parking lot, basically rode most of the place in the opposite direction I’m used to, but then again that was the first time I’ve been there since 2017 so it was all like new… Some trails had new (to me) flow sections, with bermed turns and little jumps, and I managed to get some pathetic air on one — and my Garmin chirped at me!
We all went back to J&R’s place for beer & burgers afterward; it was a later night than I expected but when I got home I confirmed it: the sound was because of a jump, and the jump location and stats were all recorded with the ride data.
Saturday’s wake-up was a little rough, but there I was, back at Nox for a morning ride with my friend Ted. We were joined by a friend of his and did a brisk-but-not-crazy standard loop — it was a hot day and I was tired, and I think they took pity on me.
When we got to the little dirt jumps I did some cautious experimenting and got three more beeps out of the Garmin… I think I’ll need to keep a tight rein on this, otherwise I’m going to hurt myself chasing that bell.
I did a Sals ride with Renee last week, and had a really tough time: I felt almost incompetent, screwing up all sorts of easy things and just not feeling strong or competent. I noticed that my feet would not stay clipped to the pedals, and also that the brakes were almost bottoming out, and since I knew that bad riding would never be my fault or issue, I focused on those as the sources of my problems.
The brakes were easy: I picked up new pads and replaced the worn ones. Brake replacement used to be the most tricky part of bicycle maintenance, but each new generation of braking technology has made a quantum leap in ease of use (even as the brakes themselves became more complicated), to the point where my biggest problem was not over-thinking the job. Pull out the old pads, slap in the new ones, done — they even do their own adjusting afterward.
My pedals/cleats were a different matter. I noticed that they (the cleats) were in fact worn, and even kind of broken, and I had spare cleats ready to go, but I could not get the old ones off my shoes: corrosion had welded the bolts in place, and I was practically stripping my Allen wrenches trying to loosen them. In the end I had to get new shoes — the originals were old, had multiple other issues, and they owed me nothing; this was really just the final straw. I got a new pair of Bontrager shoes and installed the cleats.
Yesterday morning I did a towpath ride, just a test run for my new stuff. Everything checked out fine, and I even did a little bit of playing around with the heart rate monitor, riding in various zones and trying to find my maximum heart rate. This meant that I was pushing pretty hard at some points, and I managed to finish the ride much quicker than usual. Bonus!
I hopped in the shower afterward, and just for laughs I stepped on the scale. This wasn’t my usual “official morning weigh-in” protocol by any means, it was a warm ride and I was probably dehydrated, but I got on the scale anyway and it read 180.0 pounds. I hadn’t seen that number in a while! I spent the rest of the day chugging water, and my weight was back up at 184 pounds this morning. Oh well…
So I’ve had my new Garmin Edge 830 fro a week or two now, and my verdict is: meh, it’s better than the 810 but it’s not magic. The map it comes with is OK, but I plan to put an OSM map on instead — I think I can do this, but I’m not going to try for a while — and there are a bunch of other features I haven’t explored yet, which don’t do much for me (yet), such as connectivity to Strava, map connectivity to TrailForks, and some new MTB measurements: Grit, Flow, and Jump.
“Jump” seems pretty straightforward, it measures something about how much or how well you jump on the bike, but “grit” and “flow” remain a mystery to me, even after reading their explanations. “Grit” apparently is a sort of trail difficulty score, based on turns and elevation changes, while “flow” senses the rider’s performance as a function of maintaining momentum. Maybe they’re useful, but they sure don’t seem to tell the whole story — where does rockiness, or trail roughness generally, come into the equation? We’re talking Pennsyltucky riding here, after all…
Anyway, for now the unit shows me where I am and where I’m going, and I can explore what I think of the other, new features over time.
I have the first part of my new Python project (reading and analyzing data from my Garmin): it can now extract (or calculate) distance, time, moving time, maximum speed, average speed, average moving speed, maximum and average heart rates, time in each heart rate zone, and calories for the overall ride and each lap, as well as a linestring of the path taken, and starting points for each lap. For now I only store some of these in my database, but I may add them later — that’s a database task. Here is the output for a ride where I hit the “lap” button a lot:
Processing /home/don/Desktop/temp_stuff/zippy_towpath.fit
Overall Ride Stats:
Start Time: 2019-03-15 18:32:22
Distance: 25.48 mi
Elapsed Time: 2:01:10 Average Speed: 12.62 mph
Moving Time: 2:01:08 Moving Average Speed 12.62 mph
Maximum Speed: 28.62 mph
Avg Heart Rate: 143 Max Heart Rate: 160
Calories: 977
Time in zones: ['0:00:00', '0:02:29', '0:53:39', '1:02:17', '0:00:00']
Estimated MET calories (dirt road): 975
Estimated MET calories (generic MTB): 1429
Lap 1 Stats:
Start Time: 2019-03-15 18:32:22
Distance: 2.55 mi
Elapsed Time: 0:10:14 Average Speed: 14.96 mph
Moving Time: 0:10:13 Moving Average Speed 14.98 mph
Maximum Speed: 28.62 mph
Avg Heart Rate: 121 Max Heart Rate: 149
Calories: 78
Time in zones: ['0:00:00', '0:02:29', '0:03:09', '0:01:49', '0:00:00']
Estimated MET calories (dirt road): 82
Estimated MET calories (generic MTB): 120
Lap 2 Stats:
Start Time: 2019-03-15 18:42:36
Distance: 3.55 mi
Elapsed Time: 0:14:22 Average Speed: 14.84 mph
Moving Time: 0:14:22 Moving Average Speed 14.84 mph
Maximum Speed: 18.94 mph
Avg Heart Rate: 156 Max Heart Rate: 160
Calories: 160
Time in zones: ['0:00:00', '0:00:00', '0:00:00', '0:14:22', '0:00:00']
Estimated MET calories (dirt road): 116
Estimated MET calories (generic MTB): 169
Lap 3 Stats:
Start Time: 2019-03-15 18:56:58
Distance: 0.40 mi
Elapsed Time: 0:01:51 Average Speed: 13.05 mph
Moving Time: 0:01:51 Moving Average Speed 13.01 mph
Maximum Speed: 15.24 mph
Avg Heart Rate: 149 Max Heart Rate: 156
Calories: 19
Time in zones: ['0:00:00', '0:00:00', '0:00:00', '0:01:51', '0:00:00']
Estimated MET calories (dirt road): 15
Estimated MET calories (generic MTB): 22
Lap 4 Stats:
Start Time: 2019-03-15 18:58:49
Distance: 3.89 mi
Elapsed Time: 0:17:07 Average Speed: 13.65 mph
Moving Time: 0:17:07 Moving Average Speed 13.64 mph
Maximum Speed: 18.08 mph
Avg Heart Rate: 145 Max Heart Rate: 153
Calories: 172
Time in zones: ['0:00:00', '0:00:00', '0:03:21', '0:13:46', '0:00:00']
Estimated MET calories (dirt road): 138
Estimated MET calories (generic MTB): 202
Lap 5 Stats:
Start Time: 2019-03-15 19:15:56
Distance: 2.33 mi
Elapsed Time: 0:10:21 Average Speed: 13.50 mph
Moving Time: 0:10:21 Moving Average Speed 13.52 mph
Maximum Speed: 18.29 mph
Avg Heart Rate: 139 Max Heart Rate: 147
Calories: 83
Time in zones: ['0:00:00', '0:00:00', '0:08:44', '0:01:37', '0:00:00']
Estimated MET calories (dirt road): 83
Estimated MET calories (generic MTB): 122
Lap 6 Stats:
Start Time: 2019-03-15 19:26:17
Distance: 5.73 mi
Elapsed Time: 0:28:15 Average Speed: 12.17 mph
Moving Time: 0:28:16 Moving Average Speed 12.16 mph
Maximum Speed: 22.55 mph
Avg Heart Rate: 142 Max Heart Rate: 152
Calories: 196
Time in zones: ['0:00:00', '0:00:00', '0:19:47', '0:08:29', '0:00:00']
Estimated MET calories (dirt road): 227
Estimated MET calories (generic MTB): 333
Lap 7 Stats:
Start Time: 2019-03-15 19:54:33
Distance: 6.32 mi
Elapsed Time: 0:32:54 Average Speed: 11.53 mph
Moving Time: 0:32:55 Moving Average Speed 11.52 mph
Maximum Speed: 14.50 mph
Avg Heart Rate: 143 Max Heart Rate: 150
Calories: 230
Time in zones: ['0:00:00', '0:00:00', '0:16:08', '0:16:47', '0:00:00']
Estimated MET calories (dirt road): 265
Estimated MET calories (generic MTB): 388
Lap 8 Stats:
Start Time: 2019-03-15 20:27:28
Distance: 0.70 mi
Elapsed Time: 0:06:05 Average Speed: 6.94 mph
Moving Time: 0:06:03 Moving Average Speed 6.99 mph
Maximum Speed: 12.39 mph
Avg Heart Rate: 146 Max Heart Rate: 158
Calories: 39
Time in zones: ['0:00:00', '0:00:00', '0:02:30', '0:03:36', '0:00:00']
Estimated MET calories (dirt road): 49
Estimated MET calories (generic MTB): 71
Done
There’s a little room for improvement — rounding errors cause a few problems with moving vs total times in some laps, for instance — but this is what I’m looking for. My next question is: do I want this to be a standalone program, with its own GUI and everything, or do I want to build a web interface?
A friend sent me a video how-to to build a 3d map the other day, and while I thought it was really cool I didn’t want to use the software in the video. I have some pretty good stuff already, I thought, and tried to find a way to do it with either GRASS or QGIS. GRASS was a bit of a bust: I really hate the interface they use for 3d, and couldn’t find much on how to drape one layer over another — it used to be easy!
QGIS wasn’t much better, but then I am a few versions behind. There is a plugin, however, which enabled me to make a 3D map website. So here’s mine:
I used the USGS topographic map from 1894, and “draped” it over the DEM I made for the Lehigh Valley cycle routing project (which DEM unfortunately has height in feet rather than meters, so the hill heights scale a bit big). The view in the picture is of Bethlehem and environs, with South Mountain and Lehigh Mountain on the left, and the Camel Hump, back when it was still Quaker Hill, in the upper right. Click the image and it’ll take you to the map website.
I noticed, when playing with that topo map, that for things like roads it doesn’t align everywhere with current maps. The map was provided with a CRS by USGS, but I suspect it was guesswork: there is no projection or datum information on the map itself. (The corners do line up exactly.) This may be because of surveying inaccuracies, back then or even for modern maps — I’m mostly using OpenStreetMap, after all — or it could be that the roads themselves were moved or straightened over the years, or they guessed wrong with the CSR. I thought it interesting then, that on the 3D map the hills and contour lines line up as well as they do: the surveyors knew where the hills were, at the very least.