• Category Archives tech talk

  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • What Is Your Quest?

    Posted on by Don

    I’ve been moving forward with the additional D&L access and amenities points for my project, but the trail sections south of Riegelsville are terra incognita, especially when it comes to trail access, so I relied on GIS to find access points: I split the road network into “trail” and “not trail” sections, and intersection points (that aren’t at bridges) made for pretty good access candidates; some closer map inspections verified a few obvious trailheads, and weeded out some things like private drives. A lot still needed to be verified via “ground truthing” though, and so the other day I went out for a ride, starting from Riegelsville, south along the towpath to Tinicum Park.

    I had my candidate points loaded in my GPS so I could see on the map when I came upon one; I could add locations I’d missed, and delete false positives as I spotted them, and by the time I was done I should have a pretty good idea of how to get on and off the trail. This method worked really well, and the only real problems were judgement calls at what seemed like private access points. (Things are a little different in Bucks County, there are some wealthy homes between the trail and the river, with their own driveways and footbridges, and while some crossings are obviously marked “Private – No Trespassing,” others were maintained, and painted, as if they were park property.) Judgement calls, and I think I made the right calls, but for the most part it didn’t matter — all these these access points were too far from any amenities to be useful.

    It was easy and pleasant work, and I took pictures on the way back:

    • The Promised Land!
    • All This Could Be Yours
    • Trailside Ferns and Flowers
    • The Delaware Canal
    • Gilbert Station in background.
    • Upstream of the power plant.
    • Turnaround Tree
    • One of these trees was my old turnaround point.

    Total distance, out and back, was about 24 miles, and the ride took about two and a half hours. I have the new access points and amenities incorporated into my map.


    tech talk the sporting life

  • Yet Another Half-Finished Magnum Opus

    Posted on by Don

    I put the D&L trail data online, formatted as a map/website that I hope could be useful to someone, as soon as I fix a few tweaks — and add the missing portion of trail. Meantime,

    Click Here

    Enjoy! I’ll soon be adding some more useful UI parts soon, and eventually I’ll get to the trail stuff south of Riegelsville.


    tech talk

  • Trail Project Update

    Posted on by Don

    I got those new amenities into my D&L trail amenities database — it was a piece of cake, once the data was cleaned up. The whole process went smoothly, even the main database report (done through Jaspersoft Studio, which can be a pain in the neck to work with) digested the new information without any glitches.

    But I couldn’t leave well enough alone after that: I decided to add the amenities between Allentown and Northampton where the trail is incomplete. My original feeling was that the trail would be too vague through here — where are the “access points,” the intersections between the trail and the wider world’s road network, if you’re riding on the roads and there is no specific trail? But I’ve been riding this section a bit more lately, since it’s not been inundated with users like other sections, and discovered that much of it is only “unfinished” in the sense that it’s not up to the specifications of the rest of the D&L; it’s perfectly rideable on a mountain bike, and actually more fun than the more polished sections. If you don’t mind riding the rougher stuff there’s not much trail missing, and the remaining road portions are remote enough that there’s no need to worry about nearby amenities.

    So, I repeated the process for the incomplete section: identify trail access points, import amenities from OpenStreetMap (and then clean them up, by far the most laborious part) and finally tie access points to amenities using that routing distance matrix script. Again, it worked like a charm.

    …and then I started thinking about what I might want for output. That Jasper Studios report is nice, but the current output is a PDF — ugh, not very net-friendly — and I thought it might be nicer to get a more straightforwardly data-oriented output, something I can massage and format as necessary in a browser, something like JSON. Unfortunately, though Jaspersoft can do JSON output, I couldn’t quite figure how to get it to do what I wanted. Postgresql, the database I’m using, has JSON capabilities of its own, and final-product-straight-from-the-database seems like a better approach, but I didn’t know much else about those capabilities, so I sat down over the past few days and doped it out. The learning curve was pretty steep, more like a brick wall, and my code is pretty convoluted but I did get it to work. Of course, this success is just a lead-in to another escalation: if I want a web page I now have to code out the rest of the stack.


    tech talk

  • Tired of the Flats

    Posted on by Don

    So I got the Iguana’s rear flat fixed pretty quickly, it seemed like probably just a puncture or pinch from the towpath. I also took the time to do some deferred maintenance; now the bike is looking good and performing well.

    I then took off the Cannondale’s front wheel, removed the tire, and found what I suspected but couldn’t originally spot: there was a tear in the sidewall. Luckily I had a spare tire waiting, so I put that on with the new tube, and — BANG! That tube popped as well. I was done for the day…

    I looked at it a few days later and the most likely cause seemed to be “operator error” when installing the tube, no other problems or defects were visible so I replaced the tube and gingerly pumped the tire, with many stops for bead inspections etc… everything was fine. I let it sit for a few days, but heard no sudden gunshots in the basement, so I took the bike for a short ride Monday. All is good.


    tech talk the sporting life

  • More Fun with Routing

    Posted on by Don

    I’m not sure why I did it, but I installed PHP and Apache on my new computer, then moved a bunch of my “internal website” stuff over from storage. Everything seemed to work pretty well, so I tried the commuter routing program — I got errors, natch.

    I looked at the error messages and realized that the pgRouting routines had changed, so my database routing functions were out of date — that led to me discover that even the newer version of PgAdmin3 doesn’t work well with my newer Postgresql version, especially when it comes to functions. So, I installed phpPgAdmin — which was also borked, and in the same way, but I was able to fix the source code. Even working properly it couldn’t do what I needed though, which was to modify my old function. I tried writing a new function through phpPgAdmin, which was extremely laborious, and basically re-wrote the original, broken function, so now I had two useless functions that I couldn’t modify. Ugggh, time for bed.

    I woke up this morning and got it done old-school, writing a SQL script to define the function and running that from the command line. Presto, now I have a working function, and a working commuter routing program. Bonus: the new version of pgRouting is much faster (though that could be the new computer), and some routing errors are now fixed. Wish you could see it!


    tech talk 📎and tagged

  • Fun With Alice And Bob

    Posted on by Don

    I moved my SSH and PGP certificates to my new machine, and setting them up meant visiting a few cryptographic sites (to remember how to do it, and also just because). That means I spent a lot of time in that world, where “Alice” and “Bob” (and so on, alphabetically) are the main actors in usage scenarios: Alice sends a message to Bob, but maybe Clara intercepted it, how does Bob know the message is secure? Well, Alice encrypts her message and signs it with… Tedious, tedious, but now it’s done, and I can securely access my online accounts as well as send/receive encrypted email, which is sort of like being the only person with a phone — who can you call? — but still: I once thought it was cool, and now I have it back.

    Auld Acquaintance

    This past weekend was sort of the end of the holiday season for us: we saw friends we’d missed during the holidays. Friday night we did some bar-hopping with Doug & Lori, then Saturday was a “Festivus party” at Eric & Kris’s place (where we saw D&L again) lots of catching up and the usual merriment, it was a really nice night.

    Movie Night

    We saw “Uncut Gems” the other night. I don’t recommend it: the acting and the cinematography were all good, but the characters (especially the protagonist) were annoying and unsympathetic, and the plot, as Anne put it, was “a 2-hour panic attack.” Ugh.


    cultural ramblings tech talk

  • Not Quite The Borg

    Posted on by Don

    One more piece of software I added: Jaspersoft Studios, a program to design and generate reports from various data sources. I use this because — unlike other report programs I have tried — it works well with PostgreSQL, the database I like to use, but it can be a bear to use since it has a lot of moving parts.

    There are two ways to use Jasper Reports: one is as a standalone program, and the other is as a plug-in to the Eclipse IDE, which I happen to like using, and which I use nowadays for almost all my coding. I downloaded the standalone program first, but it looked so much like an Eclipse product it reminded me that there was a plug-in — this would be just one more thing that I could run from my single IDE to rule them all. So, I installed the plugin.

    What I got, plug-in and standalone package, was version 6.10, and it really looked beautiful. Unfortunately, as soon as I started to use it, I found that it was too broken to use… Jasper Reports is mostly like a graphical page layout program, where text and other elements can be moved and resized on the page, but the elements here wouldn’t move or resize. Which is bad.

    It took me a day to find information on the problem. I never did find a solution but I did find a workaround: just use an earlier version. That’s what I did, installing standalone version 6.8; Eclipse will have to wait until I learn how to add older plug-ins.

    Anyway, the program works fine now, and still looks beautiful. I used it today to make a table of bike rides from my database, natch.

    Reading

    I just got the latest Jeff VanDermeer novel, Dead Astronauts. This is set in the same story-universe as his Borne and A Strange Bird, and some of his short stories, but it is supposedly far more complex and experimental. More on this after I get through the first page…


    cultural ramblings tech talk

  • Garmin: Time In Heart Rate Zone

    Posted on by Don

    I guess the honeymoon is over, I’m starting to find things I don’t like about my Edge 830. In its defense, I think that this was actually caused by a Garmin update…

    I noticed this a little while ago: when I rode with my heart rate monitor, the data would be recorded correctly, except that “time in heart rate zones” would be completely off.

    (Heart rate zones are pretty much what they sound like: intervals of heart rate beats per minute, like 80-105 BPM, 105-130, etc, that indicate exercise intensity. There are typically five zones, which is how have mine set up: Zones 1 and 2 are for warming up and easier riding, I spend most of my time in Zone 3 just below my anaerobic threshold, Zone 4 is when I’m pushing it, and I don’t hit Zone 5 unless I’m going all out. So, mostly in Zone 3, a bit in Zones 4, 2, and 1, and almost nothing in Zone 5.)

    That’s how it is usually — but what I’d been seeing lately was almost all of the ride in Zone 5, with maybe a few seconds in some of the others. When I hovered over the time in zones on the Garmin Connect website — hovering should reveal the high and low heart rates defining each zone — I found that Zone 5 was set as anything over 3 beats per minute, and the rest were basically set to nonsense.

    I checked Garmin Connect on my phone — for Linux, GC on the web is sort of crippled so I use the phone app to access the device — and heart rate zones were set correctly for both generic sports and also for cycling. I looked up, on the Garmin site, how to set heart rate, and I could have sworn that the only way to do it was through Garmin Connect, but they had instructions for doing it directly on the unit. I follow those instructions, and sure enough, there’s a setting for heart rate zones; there is only the one option, no separate sets of zones for different sports, and the zones in the setting are the nonsense I saw before.

    I set those zones correctly, and they worked fine on tonight’s ride.

    I’m not sure what happened, but I’d guess that Garmin pushed some update that added the heart rate zones setting to the unit, but screwed up the implementation so that the unit no longer sync’ed with the data on Garmin Connect.


    tech talk

  • The Long Transition

    Posted on by Don

    Well, the new laptop has been here for over a week now, long enough for me to make some first impressions, while moving myself over to the new machine and customizing it (as well as I can) to fit my needs…

    The first thing I did was mess with the desktop. At first I tried to live with the original desktop environment, which is based on the Gnome environment but built by System76 as part of their own Pop!-OS distribution. I managed to get a few things the way I like them, but it still just seemed unpleasant so I installed Cinnamon. A whole bunch of customizations later… it’s not 100% where I want it, but I’m pretty close.

    My next step was to install the programs I use (that don’t come automatically installed), like PostgGIS, QGIS, R, GRAMPS and Eclipse. That was pretty easy, actually, and actually was done before I finished setting up my desktop. (I may never “finish setting up my desktop.”) I got Dropbox working, which is nice to have again — it stopped supporting my old laptop’s filesystem a while ago — and that brought me into the next phase of the transition on its own.

    That next phase was moving my data files – QGIS and GRASS projects, documents and spreadsheets, programs I wrote, photos, music, my databases (yikes!) — over to the new computer. Dropbox started by synchronizing with my new laptop, meaning it had to put several gigabytes onto my new machine; it did this automatically but the process took days at our slow internet connection rate, especially since I kept pausing it so we could watch movies or do other internet things.

    As for the manual transfers: I moved over my PostgreSQL databases as a big first step, thinking it would be the most difficult thing. It really wasn’t; the backup and restore utilities worked flawlessly, even with all the PostGIS extensions. Once the databases were moved over, I moved all the GIS stuff that I intend to keep working on. QGIS needed a little help finding my data resources in their new locations, but that also went pretty well. Moving the Eclipse projects, and my music, and my GRAMPS family tree data, were also fairly easy. I have yet to try moving the photos, with all their metadata, but that’s coming. (I have other databases running on MySql, but I have not even installed MySQL on this machine yet.)

    Much of what’s on the old laptop I don’t really want on the new one, so once this data-transfer process is done, I will do one final backup from there to the NAS drive, then shut the laptop down and put it in a closet somewhere.

    Actually using my stuff on the new machine? That’s been its own set of problems, mostly having to do with how far behind I had been with upgrades on the old one. I had been planning to make the jump from Python 2 to Python 3, which means rewriting my old Python scripts to suit the changes. That’s a bit tedious but the process is not too hard, for purely Python things at least. For QGIS, which made that transition (and a lot of other changes) as part of their major upgrade to version 3, my algorithm scripts were no longer even recognizable as scripts by the program — the system is completely different. I buckled down and ported one script over to the new system — and it worked! — but I see I have my work cut out for me.

    The new laptop, as a physical thing, is awesome: it’s good-looking, and light (though it seems a bit flimsy compared to the older, heavier laptop), and it seems really fast, especially loading software. I was worried about the keyboard, but it seems fine — I’m still getting used to it though, and now when I go back to the Dell I’m not good with that anymore either. Transitions…


    tech talk

  • Infrastructure Week

    Posted on by Don

    Busy week:

    • I’m writing this on my new laptop, which came Wednesday — I’ve been setting it up and transferring files ever since.
    • Last Sunday we looked at new ranges, and bought one Monday, which should arrive tomorrow.
    • Tuesday we bought a new storm door, which Mark D will be installing when he gets the chance.
    • I also looked into the stereo and got it working (or, at least, “working,” since one whole side just does not produce sound), but it probably could use replacing. That’s going to have to wait for more research.
    • Also up on deck: our switch to new cable, but that also has to hold for more research — if we go with our own modem and router, another purchase.

    But things are happening on the infrastructure front!

    Next up this afternoon is brewing.


    day by day tech talk