• Category Archives tech talk

  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • More Fun with Routing

    Posted on by Don

    I’m not sure why I did it, but I installed PHP and Apache on my new computer, then moved a bunch of my “internal website” stuff over from storage. Everything seemed to work pretty well, so I tried the commuter routing program — I got errors, natch.

    I looked at the error messages and realized that the pgRouting routines had changed, so my database routing functions were out of date — that led to me discover that even the newer version of PgAdmin3 doesn’t work well with my newer Postgresql version, especially when it comes to functions. So, I installed phpPgAdmin — which was also borked, and in the same way, but I was able to fix the source code. Even working properly it couldn’t do what I needed though, which was to modify my old function. I tried writing a new function through phpPgAdmin, which was extremely laborious, and basically re-wrote the original, broken function, so now I had two useless functions that I couldn’t modify. Ugggh, time for bed.

    I woke up this morning and got it done old-school, writing a SQL script to define the function and running that from the command line. Presto, now I have a working function, and a working commuter routing program. Bonus: the new version of pgRouting is much faster (though that could be the new computer), and some routing errors are now fixed. Wish you could see it!


    tech talk 📎and tagged

  • Fun With Alice And Bob

    Posted on by Don

    I moved my SSH and PGP certificates to my new machine, and setting them up meant visiting a few cryptographic sites (to remember how to do it, and also just because). That means I spent a lot of time in that world, where “Alice” and “Bob” (and so on, alphabetically) are the main actors in usage scenarios: Alice sends a message to Bob, but maybe Clara intercepted it, how does Bob know the message is secure? Well, Alice encrypts her message and signs it with… Tedious, tedious, but now it’s done, and I can securely access my online accounts as well as send/receive encrypted email, which is sort of like being the only person with a phone — who can you call? — but still: I once thought it was cool, and now I have it back.

    Auld Acquaintance

    This past weekend was sort of the end of the holiday season for us: we saw friends we’d missed during the holidays. Friday night we did some bar-hopping with Doug & Lori, then Saturday was a “Festivus party” at Eric & Kris’s place (where we saw D&L again) lots of catching up and the usual merriment, it was a really nice night.

    Movie Night

    We saw “Uncut Gems” the other night. I don’t recommend it: the acting and the cinematography were all good, but the characters (especially the protagonist) were annoying and unsympathetic, and the plot, as Anne put it, was “a 2-hour panic attack.” Ugh.


    cultural ramblings tech talk

  • Not Quite The Borg

    Posted on by Don

    One more piece of software I added: Jaspersoft Studios, a program to design and generate reports from various data sources. I use this because — unlike other report programs I have tried — it works well with PostgreSQL, the database I like to use, but it can be a bear to use since it has a lot of moving parts.

    There are two ways to use Jasper Reports: one is as a standalone program, and the other is as a plug-in to the Eclipse IDE, which I happen to like using, and which I use nowadays for almost all my coding. I downloaded the standalone program first, but it looked so much like an Eclipse product it reminded me that there was a plug-in — this would be just one more thing that I could run from my single IDE to rule them all. So, I installed the plugin.

    What I got, plug-in and standalone package, was version 6.10, and it really looked beautiful. Unfortunately, as soon as I started to use it, I found that it was too broken to use… Jasper Reports is mostly like a graphical page layout program, where text and other elements can be moved and resized on the page, but the elements here wouldn’t move or resize. Which is bad.

    It took me a day to find information on the problem. I never did find a solution but I did find a workaround: just use an earlier version. That’s what I did, installing standalone version 6.8; Eclipse will have to wait until I learn how to add older plug-ins.

    Anyway, the program works fine now, and still looks beautiful. I used it today to make a table of bike rides from my database, natch.

    Reading

    I just got the latest Jeff VanDermeer novel, Dead Astronauts. This is set in the same story-universe as his Borne and A Strange Bird, and some of his short stories, but it is supposedly far more complex and experimental. More on this after I get through the first page…


    cultural ramblings tech talk

  • Garmin: Time In Heart Rate Zone

    Posted on by Don

    I guess the honeymoon is over, I’m starting to find things I don’t like about my Edge 830. In its defense, I think that this was actually caused by a Garmin update…

    I noticed this a little while ago: when I rode with my heart rate monitor, the data would be recorded correctly, except that “time in heart rate zones” would be completely off.

    (Heart rate zones are pretty much what they sound like: intervals of heart rate beats per minute, like 80-105 BPM, 105-130, etc, that indicate exercise intensity. There are typically five zones, which is how have mine set up: Zones 1 and 2 are for warming up and easier riding, I spend most of my time in Zone 3 just below my anaerobic threshold, Zone 4 is when I’m pushing it, and I don’t hit Zone 5 unless I’m going all out. So, mostly in Zone 3, a bit in Zones 4, 2, and 1, and almost nothing in Zone 5.)

    That’s how it is usually — but what I’d been seeing lately was almost all of the ride in Zone 5, with maybe a few seconds in some of the others. When I hovered over the time in zones on the Garmin Connect website — hovering should reveal the high and low heart rates defining each zone — I found that Zone 5 was set as anything over 3 beats per minute, and the rest were basically set to nonsense.

    I checked Garmin Connect on my phone — for Linux, GC on the web is sort of crippled so I use the phone app to access the device — and heart rate zones were set correctly for both generic sports and also for cycling. I looked up, on the Garmin site, how to set heart rate, and I could have sworn that the only way to do it was through Garmin Connect, but they had instructions for doing it directly on the unit. I follow those instructions, and sure enough, there’s a setting for heart rate zones; there is only the one option, no separate sets of zones for different sports, and the zones in the setting are the nonsense I saw before.

    I set those zones correctly, and they worked fine on tonight’s ride.

    I’m not sure what happened, but I’d guess that Garmin pushed some update that added the heart rate zones setting to the unit, but screwed up the implementation so that the unit no longer sync’ed with the data on Garmin Connect.


    tech talk

  • The Long Transition

    Posted on by Don

    Well, the new laptop has been here for over a week now, long enough for me to make some first impressions, while moving myself over to the new machine and customizing it (as well as I can) to fit my needs…

    The first thing I did was mess with the desktop. At first I tried to live with the original desktop environment, which is based on the Gnome environment but built by System76 as part of their own Pop!-OS distribution. I managed to get a few things the way I like them, but it still just seemed unpleasant so I installed Cinnamon. A whole bunch of customizations later… it’s not 100% where I want it, but I’m pretty close.

    My next step was to install the programs I use (that don’t come automatically installed), like PostgGIS, QGIS, R, GRAMPS and Eclipse. That was pretty easy, actually, and actually was done before I finished setting up my desktop. (I may never “finish setting up my desktop.”) I got Dropbox working, which is nice to have again — it stopped supporting my old laptop’s filesystem a while ago — and that brought me into the next phase of the transition on its own.

    That next phase was moving my data files – QGIS and GRASS projects, documents and spreadsheets, programs I wrote, photos, music, my databases (yikes!) — over to the new computer. Dropbox started by synchronizing with my new laptop, meaning it had to put several gigabytes onto my new machine; it did this automatically but the process took days at our slow internet connection rate, especially since I kept pausing it so we could watch movies or do other internet things.

    As for the manual transfers: I moved over my PostgreSQL databases as a big first step, thinking it would be the most difficult thing. It really wasn’t; the backup and restore utilities worked flawlessly, even with all the PostGIS extensions. Once the databases were moved over, I moved all the GIS stuff that I intend to keep working on. QGIS needed a little help finding my data resources in their new locations, but that also went pretty well. Moving the Eclipse projects, and my music, and my GRAMPS family tree data, were also fairly easy. I have yet to try moving the photos, with all their metadata, but that’s coming. (I have other databases running on MySql, but I have not even installed MySQL on this machine yet.)

    Much of what’s on the old laptop I don’t really want on the new one, so once this data-transfer process is done, I will do one final backup from there to the NAS drive, then shut the laptop down and put it in a closet somewhere.

    Actually using my stuff on the new machine? That’s been its own set of problems, mostly having to do with how far behind I had been with upgrades on the old one. I had been planning to make the jump from Python 2 to Python 3, which means rewriting my old Python scripts to suit the changes. That’s a bit tedious but the process is not too hard, for purely Python things at least. For QGIS, which made that transition (and a lot of other changes) as part of their major upgrade to version 3, my algorithm scripts were no longer even recognizable as scripts by the program — the system is completely different. I buckled down and ported one script over to the new system — and it worked! — but I see I have my work cut out for me.

    The new laptop, as a physical thing, is awesome: it’s good-looking, and light (though it seems a bit flimsy compared to the older, heavier laptop), and it seems really fast, especially loading software. I was worried about the keyboard, but it seems fine — I’m still getting used to it though, and now when I go back to the Dell I’m not good with that anymore either. Transitions…


    tech talk

  • Infrastructure Week

    Posted on by Don

    Busy week:

    • I’m writing this on my new laptop, which came Wednesday — I’ve been setting it up and transferring files ever since.
    • Last Sunday we looked at new ranges, and bought one Monday, which should arrive tomorrow.
    • Tuesday we bought a new storm door, which Mark D will be installing when he gets the chance.
    • I also looked into the stereo and got it working (or, at least, “working,” since one whole side just does not produce sound), but it probably could use replacing. That’s going to have to wait for more research.
    • Also up on deck: our switch to new cable, but that also has to hold for more research — if we go with our own modem and router, another purchase.

    But things are happening on the infrastructure front!

    Next up this afternoon is brewing.


    day by day tech talk

  • Tech Updates

    Posted on by Don

    I just thought I’d post an update on my new computer: it’s been assembled, it’s been shipped and it’s due to arrive here this Wednesday. I can’t wait!

    Meantime, I was looking into what was going wrong with my GeoPackages, and I found out that there are problems with (my now slightly outdated versions of) QGIS and Spatialite that prevent GeoPackage spatial queries from working. So it wasn’t me!


    tech talk

  • Project Drift

    Posted on by Don

    I’ve done a few more Road Scholar gigs this year, and my co-guide and I both feel that the ride choices could be improved, mainly by doing more bike paths and rail-trails, and doing less actual road riding. This would avoid the biggest issues we face (traffic and hills), and maybe allow the rides to be a bit longer and more enjoyable.

    Meantime, I’d noticed a tendency, among our van drivers, to use Google Maps to navigate our pick-up, drop-off and other van access points. This is I think a good thing, but it’s led to map searches finding the wrong drop-off point — nearby features rather than the specific location we use. It works well enough that “OK, turn left here and pull into that parking lot” will get us there once we’re close enough, but navigating to an actual position (a given latitude and longitude, for instance) would work much better.

    Finally, I thought it would be good to have an official repository somewhere, of the rides: their official routes (I use GPS to navigate on the rides) as well as waypoints, like lunch spots, points of interest along the ride, and those pick-up and drop-off points. Ideally, I would be able to load a ride into my GPS and have all info for the ride at my fingertips.

    These all coalesced in my mind into the Great Big Ride Database GIS Project. The project would be made of three parts: storage of rides (official or otherwise) and waypoints into a ride database, transfer of rides/waypoints to and from my GPS, and analysis of the ride data.

    First Steps, and Revolting Developments

    I started by keeping “official versions” of our rides on RideWithGPS, and I would download them as GPX files onto my Garmin when I needed them. This would only take care of the route itself, however; I thought that there was also a need to maintain a list of waypoints associated with each route, so I decided to build some kind of database to hold routes and their waypoints.

    Since I would like to be able to just hand over the ride information in some file format, my first attempt was to build the database as a GeoPackage file. This actually worked pretty well, when my plan was just to stuff the data into storage. But then, my plans started to morph: I needed to actually analyze the data (with a spatial query) to generate info I needed. The GeoPackage file should have been able to handle this, but I think I must have done something wrong back when I installed the underlying GeoPackage/SpatiaLite libraries, or I was doing something wrong now, but I just couldn’t get any spatial functions to work. After frustrating myself for a while I just moved the database over to PostGIS. My project was changing, but at least it worked.

    So at this point, I started looking at the problem of getting the point data to places where I could use it — like onto and off of my Garmin. I collected a bunch of the waypoints as “saved locations” on my GPS, but then I couldn’t find any good way to export or upload them. (The Google tells me that Garmin apparently has some Windows programs that can manage waypoints, but that does me no good.)

    I eventually dropped back and punted by writing a Python script. I scrounged around inside my Garmin and found a file called Locations.fit that seemed to be where the saved locations were stored, and used that fitparse library to rummage inside the FIT file, eventually figuring out the (undocumented) structure used to store waypoints. I could now export the waypoints into a QGIS layer, then I managed to realize that I could import the waypoints to my GPS via a GPX file in the same way I could import rides via GPX, and could even combine waypoints with the ride trackpoints in the same file for importing. Major breakthrough! — though the Garmin seemingly ignores all waypoint information (symbology, comment) except the name.

    So things are now a bit different than how I first planned it, but I have a system that works. Next up: evaluating potential routes.


    tech talk 📎and tagged

  • New Purchase

    Posted on by Don

    The laptop is starting to really show ominous signs of impending demise — screen frequently goes static-ey or blank on startup, wi-fi and bluetooth unexpectedly crap out, and it no longer can wake up from sleep mode — things that started happening a few months ago but are now getting to be almost constant, or at least daily, occurrences.

    I’ve had my laptop (a Dell XPS M1530) for 10 years now; it was cutting edge when I bought, it but a decade of advancement has left it so far behind that some software can’t even be updated anymore. But I love this computer even so, and if it weren’t on the verge of crapping out I wouldn’t even think of replacement. It’s lasted probably twice as long as it should though, and owes me nothing at this point, as long as it lasts until the new one shows up…

    So anyway, I went online yesterday and bought my new one, a System76 Darter Pro. I wanted another 15.6″ screen, I think that’s the sweet spot for laptops, and I also wanted Linux pre-installed. I was also looking for reasonable battery life, light weight etc, and I also wanted something with a bit of pep, even by current standards. I think I compromised a bit by getting an i5 rather than an i7 chip, but I turned around and upgraded to 16G of RAM and a 500G NVMe solid-state drive, improvements at the likely performance bottlenecks, so I think I’ll get a good bang for my buck. It should arrive in just under two weeks.


    tech talk

  • Old Paint, Like The “Mary Ellen Carter”

    Posted on by Don

    I finally made good on the promise I made to Anne, to my brother-in-law Ray — to anyone who would listen, really — to get a new headset for the Iguana, after that bike-pukes-ball-bearings debacle at Mauch Chunk Lake. I ordered the headset (from Saucon Valley Bikes) last week, brought the bike in a few days ago for them to install it, and picked it up this afternoon. I haven’t done a test ride yet, but it feels great.

    Once I got it home I took apart the rear grip shifter and lubed it, and replaced the shifter cable and housing while I was at it. This took longer than I expected, since I forgot how to work with grip shifters, but eventually it all came together — I have now resolved two decade-old problem areas.

    Right in the nick of time too, because next week I’m back to the Road Scholar tours, and the Iguana is my go-to bike for these rides because it can carry panniers.


    tech talk