• Category Archives tech talk

  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • New Bike!

    Posted on by Don

    My writer’s block continues, but I thought I’d pop in to announce that I finally got my new bike, a Kona Sutra SE. This is a bike seriously designed for touring: fenders, a triple ring, (mechanical) disk brakes, bullet-proof tires, bar-end shifters, a Brooks saddle, and a rear rack, with frame attachments for a front rack as well — and oh yeah, steel is real, baby!

    bicycle
    My New Kona Sutra SE

    I picked it up Thursday night from Cutters Bike Shop, where it had just arrived; they knew I wanted one, and they called a week or so ago to say one was on the way, and then it was there at the shop, and so was I… I gave it a test ride with panniers, then rode it home — it doesn’t fit on the roof rack. Yesterday was its real maiden voyage, a bakery ride up to Nazareth.


    tech talk the sporting life 📎and tagged

  • Native Paths Update

    Posted on by Don

    I kept at it, and am now about a quarter of the way through the trails — the motorway parts, at least — in my Indian Paths of Pennsylvania project. I have a pretty good idea of how the book is organized now, and came up with a pretty decent workflow that gets me through a single path in just under an hour. I do one or two a day. It’s pretty easy to get absorbed, trying to find the tiny old roads and landmarks based on their descriptions in the book, and I’ve been totally sucked into the history of that Colonial-Revolutionary era. (I picked up Mason & Dixon again, since it goes right through the middle of that time and place.)

    I also think there will be some epic rides this summer, based on these routes — I’ve been drooling over some of the scenes I see in Google Street View.


    day by day looking back tech talk 📎and tagged

  • N+1

    Posted on by Don

    I knew it would come to this sooner or later — I’m in the market for a new bike. There’s nothing wrong with any of my other bikes, but none of them are really touring bikes, and I’ll be joining Anne on a trip this summer. Fully self-supported, front and rear panniers, camping in the Rockies — the works. (I hated “touring” every time I’ve ever done it, but I suspect that that’s really an equipment issue — I do enjoy our towpath “bikepacking” trips.)

    Anyway, I’ve been doing some research, and what I think I need is:

    • a bike with a “touring” frame — low bottom bracket, longer wheelbase and chainstays,
    • a triple chainring,
    • mechanical disk brakes, and
    • a few add-ons that would be nice if they came stock, like racks and fenders.

    There are a few bikes that I think might fit the bill, ones I found in several “best touring bike” listicles, namely the Trek 520, the Kona Sutra SE, and the Surly Disk Trucker. The Trek looks to be impossible to find anywhere right now, and the Surly only seems available (sight unseen) via the Internet, but I found a Sutra SE at a local bike shop, and it looks like my size. I need to do a bit more search and research — I’m also looking among the world’s used bikes — but I think I can already see how this will shake out.


    tech talk the sporting life

  • Foiled Again!

    Posted on by Don

    I have a love-hate relationship with Paul A. W. Wallace’s Indian Paths of Pennsylvania. I love reading the individual chapters on each path — their descriptions, and the accounts of them in the letters and diary entries of early Colonial explorers, but any hard look at the specifics and the trails themselves become frustratingly vague. This is all the more frustrating because the information looks specific and authoritative enough, until you take that close look…

    Some of this is because the original information is vague — nobody was tracking their steps with a GPS back then — so the actual trail location is not perfectly known, and partly it’s because the trails themselves are long gone (though some are at least partly followed by modern roads), so it’s hard to search them out without trespassing, but there also just seems to be some missing ingredient needed to define a trail network.

    A few years ago I thought that this last part could be solved with a little bit of GIS detective work, so I started a QGIS project to define the trails and see about building a network, but I sort of ran out of steam — I basically foundered on the vagueness of the trail descriptions. I did one or two in the Lehigh Valley, and realized that the sleuthing needed was a lot more laborious than simple data entry, and the project languished after those first few paths.

    I was thinking about all this again recently, and realized that there is a critical first step I ignored: the book serves primarily as an automotive guide, with detailed instructions for driving in the vicinity of each path. I also thought that if I broke the task down to a set of database tables, I could link these auto routes to their various paths and book chapters . (Some trail chapters actually describe multiple trails and subtrails, while some motorway descriptions continue across multiple chapters, so many-to-many relationships abound but that’s what databases are for. Furthermore, most of the trail chapters have a start and an endpoint, yet more data I can use to cross reference.)

    This scheme fell apart within the first few trails. The very first trail, the “Allegheny Path,” has Philadelphia as the start point and “Pittsburg and Kittanning” as the endpoint — so which is the endpoint? Apparently neither, because the trail is only described as far as Harrisburg; the “Allegheny Path” chapter ends with references to several other trails (different chapters, in other words) heading West from Harrisburg as possible continuations. So OK, I can deal with this: my endpoints are really Philadelphia and Harrisburg, and I’ll stuff the rest of the info into my “description” column. (There is a second path listed in that first chapter, but it is little more than a historical aside and a reference to another path/chapter. This is going to get tricky.)

    Luckily the motorway for the Allegheny Path is easy to follow. I used an open routing plugin to follow along a bunch of control points, and voilá I had my linestring. This ain’t so bad!

    The very next chapter, I ran into motorway difficulties: the route description made no sense. Either the routes were not prepared with adequate ground-truthing (unlikely, though I was starting to feel uncharitable), or the roads (and their designations) had changed at some point in the 55 years since the book came out. This seems the more likely explanation, since I-80 goes right through the area in question, was only finished in 1970, and probably changed a lot of things in its wake. I actually found the Wikipedia article on the Bald Eagle Creek Path more useful.

    So I’m back to deciphering and making judgement calls rather than strictly converting the information from one format to another, even for these road descriptions. I didn’t expect this project to be done in an afternoon, or even a week or so, but “going to take forever because I’m not really working on it” is now closer to my expectation.

    (Note: I found that someone already took these paths and put them into a GIS, but it’s on PA-Share and that’s proved difficult to work with — and deliberately limited, unless you pay — so far. We’ll see…)


    looking back tech talk 📎and tagged

  • Oh No Not That Again (Part 2 of 2)

    Posted on by Don

    So I was playing with the commuter mapping program the other day after doing some simple maintenance, just finding routes from here to there, and it started bothering me again that I could not route onto the towpath from Sand Island — the network was incorrect, it had no intersection from Main Street onto the path.

    I get my road data from OpenStreetmap, and I know that, in OSM, the trail is properly connected at Sand Island — I fixed that myself years ago, but never went through the rigamarole of updating and rebuilding my network. It didn’t seem worth the work for such a small change. There is another way to make that change though: I could modify my existing network, but that always seemed like it would be even more complicated and difficult than rebuilding from scratch.

    But would it be? The task really boils down to two things: adding a node where I want the new intersection to be, and then splitting the newly intersecting roads in two at the intersection point. Adding a node is easy enough, but splitting a road has a lot of moving parts — each of the two new road segments has to be assigned about 30 attributes, some of which they can inherit directly from the original road, others basically pro-rated from the original road based on the new road segment lengths, and yet others related to connecting the new road segments to the new node. It’s straightforward, but there are a lot of small, tedious calculations to perform and keep track of. Sounds like a job for the computer…

    What I did was write a PostgreSQL function that takes the node and the road, and returns two new road segments. I also wrote a wrapper script to update the network by calling this function. (I decided to just add my new nodes “by hand.”) It mostly works, though in one test case it didn’t split the road exactly where I thought it should (no idea why), and the new network routes like a champ.

    This isn’t a substitute for rebuilding the network: this is a quick fix for a small problem, and the the pro-rated attributes especially are a hack, an approximation; I can easily see situations where pro-rating say, ascent/descent data would be inaccurate. But this is fine for now.


    tech talk 📎and tagged

  • Oh No Not That Again (Part 1 of 2)

    Posted on by Don

    I’ve been looking at my Lehigh Valley bike commuter routing project again.

    I decided to update the recommended routes with additions based on some of our recent CAT rides, and found that the line geometries representing the various routes were missing. It’s no biggie, some things didn’t survive those destructive “upgrades” I did a few years ago, and the actual recommendation info is stored in among the road network data anyway.

    But, I still had the old routes as GeoJSON files, and it’s easier to work with them as geometries in their own right than as attributes on the road network, so I added them back into the database. Then I added that new route (Cedar Street, which parallels Union Boulevard but is much quieter), and used it to update the network. Piece of cake!

    I also decided to tackle the problem of updating the network paths themselves, which is not so much a piece of cake. I get the roads from OpenStreetmap, and there are mostly automated tools to build a routing network from OSM road data, and that’s followed by a whole lot of additional data massage to put it in the form I use. But the underlying OSM data isn’t always accurate — roads don’t go where they are supposed to, intersections don’t actually connect, that sort of thing. I would find a lot of this out after building the network, but the task of editing the network, once it’s built, is so onerous that my preferred method has been to fix the issues in OpenStreetmap, then just download the roads and rebuild the whole network from scratch — also onerous, but slightly less so.

    Anyway, I planned to make this a part of the usual site maintenance if this ever went live: maybe once a month I would download the OSM roads, rebuild the network, and then install all my extra stuff, and in between these upgrades I would fix OpenStreetmap whenever I found a problem.

    The last time, and in fact the only time, I ever went through this updating process, was October 2018. I did some serious cleanup on OSM before that, so the map was in pretty good shape, but I got an embarrassing surprise when I demo’ed it to John R (an actual computer professional), who was thinking of commuting to Easton via the towpath. I’d just added offroad path options, and I was eager to show John my new toy, but the program refused to route onto the towpath at Sand Island — there was a missing intersection! A classic case of “broken demo.”

    The need for (and my interest in) the routing program faded not long after that, so, although I cleaned up the offending roads and paths on Sand Island within OpenStreetmap, I never did download any newer road versions. And that’s how it sat for three years, until this week…

    (to be continued)


    day by day tech talk 📎and tagged

  • Little Hacks

    Posted on by Don

    Just a few simple solutions to life’s little computer problems…

    WordPress

    When I post photos on my blog (in photo galleries, like the one here), if the photos have captions they show up when you hover over the photo thumbs. So far so good, but when the caption appeared it was at the top of the thumb, as opposed to appearing at the bottom, and the space used by the caption was much bigger than it needed to be, almost covering the thumbnail image. This has been driving me nuts for about a year; I assumed it had something to do with my offbeat WordPress theme, but yesterday I did some Googling and found people complaining about it, starting abut a year ago — apparently the offending caption styling was related to some WordPress update.

    I found several styling fixes posted online, but none seemed to work — it seems that a later WP update might have broken these fixes as well. The nature of these posted solutions gave me an idea though: I used Mozilla’s “inspect element” to get the CSS elements responsible for the captions, and used what I found to come up with a modification that would properly style my gallery figure captions. I added that to my theme’s custom CSS and it worked like a charm.

    Here is the new custom CSS that did the job, in case I need it again, or others need it and find themselves here:

    .wp-block-gallery.aligncenter .blocks-gallery-item figcaption, .wp-block-gallery.alignleft .blocks-gallery-item figcaption, .wp-block-gallery.alignright .blocks-gallery-item figcaption, .wp-block-gallery .blocks-gallery-item figcaption, .gallery figcaption {
     min-height: 5%;
     left: 0px;
     top: auto;
     height: auto;
     bottom: 0px;
     text-align: center;
     margin: auto;
}

    Fonts

    Fonts have always been a mysterious pain in the butt for me. It doesn’t help that I have hundreds of fonts installed by default, many with similar names and meant mainly for non-Latin alphabets. I was experimenting, searching for the right look for a document, and had to scroll through hundreds of useless fonts while trying to find something to suit me — very frustrating!

    I did some Googling, and I found I could just delete font files to get rid of them — but that’s a non-solution since I hate to throw anything away, you never know when you might need the perfect Linear B font…

    A little more research, and I eventually found and installed a simple utility called (surprise) “font manager,” which let me enable and disable fonts without deleting them. I spent some time shutting off my nuisance fonts, and now my word processing life is much simpler.


    tech talk

  • On Second Thought

    Posted on by Don

    Well, that didn’t take long…

    A few days of actually using the Input app, and I’m ready to throw it away and go back to QField, despite QField’s clunky data transfer method. The need to put my collected data into a PostGIS database (without too many hoops to jump through) is more important than I realized, and Input’s data entry UI had a few quirks that just became more unpleasant every time I used it — there was something just plain off about its text boxes and typing…

    It’s a shame too, because the Mergin update process was exactly what I wanted. (Input can also handle QR codes as data sources, something I have absolutely no use for — but hey, nerdgasm alert.) Well, QField is eventually supposed to get its own cloud service, maybe they’ll be able to upgrade their data transfer process once they have that in place. My luck, they’ll reproduce Mergin’s setup, and reproduce Mergin’s PostGIS problems along with it.


    tech talk 📎and tagged

  • Data Collection II: Input

    Posted on by Don

    Just picking up where I left off here…

    I installed and started working with the other field geo-data collection app, called Input. It has a few drawbacks (so far) compared to QField, but it’s just as easy to use on the phone, and much easier to set up and transfer data back to the home computer.

    Input is basically a phone app front end for Mergin, a cloud-based data storage service built to integrate with QGIS. Mergin stores the data that then can be synch’ed with QGIS on the desktop, or with the app. Data collection does not need to be sent to the cloud in real time if you have no data connection, but can be done later, and is “just push the button” easy, as is synching between the cloud and QGIS. (This is the biggest advantage over QField, which has a pretty clunky update workflow.)

    The biggest problem I found so far has been that the data is not easily uploaded into a PostGIS table; the project relies on GeoPackages for data storage, even on the QGIS end. (Strictly speaking, there is a way to use PostGIS, but it seems involved, and needs to use another program that I have not yet tried.)

    There is also the issue of cloud storage: a free account is limited to 100 MB, and while the actual “location and description” data is usually pretty small, all the accompanying photos, at 5-10 MB each, will quickly bump up against that limit. If I could offload the data and photos, from the cloud-based project into something else (like my computer, and PostGIS), this limit wouldn’t be so pressing, but the workflow is starting to look unwieldy again…

    (Speaking of unwieldy, there is no direct way to store basemap tiles for offline use. Again, there is a way, but it’s not straightforward, and storing the tiles will also consume a significant portion of that 100MB limit.)

    None of these problems are deal-breakers (so far), and the single advantage of easy synchronization more than makes up for the lot of them. I’ll be playing with Input/Mergin some more, but I think it’s the one I’ll decide to keep.


    tech talk 📎and tagged

  • Golem Unchained

    Posted on by Don

    I’ve been using my handy, text-based “to-do” list for a while now, but my use of it was inconsistent, waxing and waning as it occurred (or didn’t occur) to me to use it, and I also didn’t like keeping a text editor open all the time whenever I did use it. So, the other day I added a “desklet” to my desktop, one that displays (the first lines of) a text file. I set it to display my to-do list, which I now can see any time, and open for editing with a click.

    Which is all well and good, but now I find that my to-do list is staring me in the face all the time, and I’ve been using it more, and more, and more… I guess I’m more productive now, as I write down all the things I need to do, then do them and mark them completed, but it seems that the tool I added to make my life easier is starting to get the upper hand, running my life instead of the other way around.

    I have to say that things get done, though.


    day by day tech talk