• Category Archives tech talk
  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • Fun With Bash

    I got to play with shell scripts a few times over the past few weeks, thought I’d talk about them:

    Wordle Helper

    I have a script I use to help me make my guesses when I play Wordle; it basically generates a list of available words based on my results, analyzes these available words (by checking the overall letter frequency, then scoring the words based on which letters they contain), and returns the word list with their scores, sorted by word score. This is especially helpful as a process of elimination, and most especially in the first rounds — I can confirm or eliminate the most common letters and quickly narrow down my choices (well, usually).

    I decided at a certain point that multi-letter combinations might be even more useful, so I wrote another script to score words based on the frequency of two-letter combinations. This worked well enough that I wrote a three-letter scoring script, and then realized that I should just write something generic that would score based on however many letters I would want in my combinations.

    For some reason I brought my laptop on vacation, and when things got quiet I broke it out and wrote the multiple-letter scoring script — it works like a charm. What I found though, is that there are diminishing returns for the letter combinations: there is no real advantage to use more than two letters. Still, this is a script that I use every day.

    Getting SSL

    Every 90 days I have to get a new SSL certificate for this website, so I can use the more secure “https” rather than plain “http.” This was once a pretty easy process, until my (free) certificate-generating organization got taken over by someone else. I found a much more onerous (but still free) web-based method and have used it for years, but I knew there had to be a better way.

    I eventually found something called getssl, a bash script that automates the process used by that onerous web-based approach. Ufortunately it’s meant to be run on the computer where the website resides, rather than on my local machine which is what I preferred, so I never really messed with it… Then one day I just set it up and ran it, and it basically did 99% of the work, generating the certificates and storing them on my laptop. All I had to do myself was install them manually, which was pretty easy. Awesome! The whole process went from an hour to about a minute, and I’ve been using getssl for almost a year now.

    But it still bothered me that I couldn’t get the whole process automated, and then last week I found an extra feature in getssl that would get me that final one percent; the only problem was that the extra feature would need to be modified to run on my machine rather than remotely. (This modification turned out to be fairly straightforward, which was a bit of a surprise actually.) I managed to make the code change just a few days ago — again, while out of town with my laptop — and my current certificate was installed, fully automatically, using the modified feature.

    Neither of these little projects were hard, but it’s so seldom that I do anything like this anymore that I was pretty proud of my accomplishments.


  • Cancel The Exorcist

    Posted on by Don

    I’ve had a creaking near my mountain bike’s bottom bracket for a few weeks now. I was pretty sure it was either the crank or the bottom bracket itself (rather than, say, the suspension pivots), since it seemed to happen with every pedal stroke, as in an unbalanced load on the BB, rather than with the bike suspension flexing.

    I tightened the cranks — no luck, so i took them off, lubed them and reinstalled them. Still no luck, so I took them off to get to the bottom bracket, cleaned and lubed the threads and reinstalled them, and still had the squeak. Maybe my bike was possessed?

    I thought maybe I didn’t do the job well enough, so I took the cranks and BB off again, and this time I also took the rear wheel off to access the suspension grease port and pumped it full of grease. I put everything back together, and this time the creaking was gone. No idea what actually worked, but the pivots really did need the grease…

    Of course, the bikes never stay perfect for long: the MTB shifting is now a bit funky, and the touring bike had a flat tire when I checked on it. It’s always something.

    Saturday was a ride at Nockamixon with Tim C, someone I know from the Museum, and Sunday was a ride up the D&L to a coffee shop in Weissport. Hot rides, both days!


  • New Toys

    Posted on by Don

    Whelp, I’m now a kayaker…

    Anne and I planned to get kayaks as our Christmas presents to each other, and were doing our research when her brother & sister-in-law gave us two of theirs as a gift. We tried to pay them, but they refused — I think we did our part to help them clean their garage out.

    We got life jackets and paddles from them too, but we still needed some way to transport the kayaks. We did a little online searching, found some used roof racks for pretty cheap, and installed them just the other day.

    Today we put the kayaks on the roof and took them to Mauch Chunk Lake for their “New (To Us) Maiden Voyage.” Driving with them on the roof was a bit unnerving, but they did just fine. We got to the lake, got them in the water, and despite the windiness of the day, we had a great time with our new toys.

    So now we have the boats, and (some of) the infrastructure, and we’ve christened them — we are kayak people now!


  • The Things We Carry

    Posted on by Don

    Anne and I just did a section hike on the Appalachian Trail, walking the New Jersey portion with several friends. More (probably much more) on this later, but one of the things that struck me was how meditative just hiking along can be — songs, other earworms, and various musings would take up a lot of the mental landscape.

    I’d built a QGIS project before our trip, with info about our section of trail, and put it on my phone. It had some distances in miles and others in kilometers, and I was mentally converting back and forth when it struck me: a kilometer is about 0.6 miles, while a mile is about 1.6 kilometers. In other words, their reciprocals differ by one. Is that true, and if not, what number would it be true for? I worked out the quadratic equation:

        \[x^2 - x -1 = 0\]

    (hmmm, seems familiar)… and I found the answer to be:

        \[1 + \sqrt{5}\over2\]

    otherwise known as φ (phi), the golden ratio. Interesting, and a fine bit of mental bubblegum to chew on as I walked along…

    When I got home, I broke my internet fast with some Reddit, and the first thing I see there is a photo of a tee shirt, with the golden ratio written on it — meaning, in this case, the “most irrational” number, as in “I am most irrational.” Anyway, just a coincidence, but a strange and striking one.

    Click To See The Shirt

    (You can see the shirt by clicking the image.)

    The other thing I carried with me in my head? “The Alley Cat,” a perfect earworm for my hiking rhythm, though I noticed that I changed the tempo as the terrain changed.


  • Dry

    The humidity in the house has been really low lately, no surprise with all this cold winter weather. Anne and I have been struggling with dry skin and sinus issues, but for me the big issue has been its effect on my cello: as it dries, the wood shrinks slightly, which makes it play flat, and recently the pegs have come loose so it became really flat and un-tunable. I had to put drops of water on the pegs, then put the cello in its case with a damp-it to recuperate.

    The cello is fine now, and the weather has been moderating — warmer, rainy — but we still broke out the room humidifiers upstairs.


  • Even More Sals GIS Fun

    My map looked so good in the QField app that I thought it might be nice to build a web map, one that could be generally available rather than part of a very specialized app. And rather than doing it from scratch, I decided to try some of the QGIS plugins.

    My first try was with the QGIS Cloud plugin. I’d used this before (verdict: meh), but I still had an account so I decided to give it another try. Verdict is still “meh” but I did get a web map out of it, check it out here. This looks as good as my QField map, which isn’t surprising since it’s basically my original project running on QGIS Cloud’s servers, but this setup came with a lot of latency: the map takes a while to redisplay after every move or resizing. It also had some trouble showing my location when I first launched it on my phone (it worked fine on the laptop) but this problem eventually resolved itself — it might have been a permissions issue, and I might have solved it by pressing random buttons…

    The other plugin I tried is called qgis2web, which builds a local web map using the standard Leaflet or OpenLayers javascript libraries. This sounded like a great approach, but as soon as I ran the plugin, it crashed QGIS — doh!

    It turns out that qgis2web can only work with very simple feature styling: lines (for instance) can be dotted or solid but not a mix, and can only be one color, while my trails were dotted lines in one color, drawn on wider solid lines in another color…

    So, I created yet another Sals sub-project, with a subset of my map’s features (just trails and roads, streams, and trailheads) and a much-simplified symbology. Just for fun I tried building an OpenLayers map, since I’d never used OpenLayers before. It came out great, and though it doesn’t look as fancy as the QGIS Cloud map it loads/runs much faster. I put this one online as well, you can find it here.

    Meanwhile, Ben and Jenny arrived yesterday for their Christmas visit, and today we went for a hike at Sals. I used my QField app to record a bunch of marker posts — I didn’t want to turn the hike into a “Don plays with his maps” debacle, so I didn’t break out either of my new web maps. But capturing the data in QField was a snap, and incorporating it into my main project was mostly seamless, and I’d guess I now have about half of the total number of markers added.

    And that means that my two new web maps are already out of date…


  • More Fun With QGIS

    Anne and I did a hike a few days ago at Sals, very pleasant and a good workout — we got in about 6 miles — but while there I was reminded of a project I’d been meaning to get started: about a year or so ago the VMB put in numbered marker posts at trail junctions, and I had been planning for a while to document them for my Sals map.

    (These numbered posts were something I’d advocated for maybe a decade ago, when I was involved with the trails up there. My advocacy didn’t go anywhere at the time, but it is good to see that the plan eventually got implemented.)

    Logging the post locations could be done using my GPS, and I’d done a few like that some time over the past year, but I am now seeing this as another opportunity to play with QField. And that’s my new project.

    The first thing I did was to clean up my Sals map project in QGIS: I have a big mish-mash of data layers in different formats (mostly geoJSON and shapefiles) and multiple coordinate reference systems, so the first thing I did was to convert them all to the same reference system (the one used by GPS devices), and then put them all in a geopackage, a sort of portable database file. This really cleaned up my project.

    I then created a new project (that also used my new Sals geopackage), and created a second geopackage, one that will be editable in the QField app (as opposed to my official data geopackage, which will be locked down), with a layer to record the new marker posts. This basically means I will capture the data in a scratch file before moving it to the official package. This approach wouldn’t work well for updating existing data, but I think it’ll do well enough here and doesn’t leave my official data as vulnerable to field mistakes.

    So I did all that, and then went through the hoops to get my new Qfield project onto my phone. Opened it up — it looks beautiful, it actually looks better on the phone than on the laptop.


  • Obsidian Dreams

    So I haven’t been writing much, here, but I have been experimenting with some software I ran across called Obsidian. Billed as a “flexible writing app” and lauded as “a second brain,” it’s basically a markdown enabled text editor with built-in file linking structure — almost like a simplified reinvention of the Web on your local machine. It’s used as a note-taking or idea-generating app, and it’s generated a lot of hype in some circles…

    I’m currently giving it a go, using it to document all the moving parts in my Native Paths project, and also to organize my thoughts for a postmortem of my experiences with the Road Scholar program.

    I find the app pretty easy to use (for my simple purposes), and I have to say it’s really nice and well-made, it’s an actual pleasure to use but I’m struggling to see where all the hype comes from. Time will tell, and maybe I’ll see more as I use it more and explore all the other features.


  • CADOP5

    Happy Ides of March! Watch your backs, and remember: the day ain’t over ’til it’s over…

    I took a graduate-level computer-aided design course in college — way back when CAD/CAM/CAE was really not a consumer-grade product — and a good part of the course was about optimization. I happened to think about that the other day when I was idly browsing through various Python modules (the way one does), and I ran across a genetic algorithm package.

    Genetic algorithms had their own vogue a few decades back, but that long-ago was long after I left college. Still, playing with it got me to reminiscing about the the things we did way back in school, and about the program we used and our professor (Dr Michael Pappas, who was its author), so I Googled him and the program.

    Dr Pappas passed away in 2015 — I found his obituary. He was a major player in the computer-aided and biomedical engineering worlds, and was the co-inventor of the artificial knee, among other things. I knew he was a big deal back in the day, but I guess I never realized how big a deal he was. (He had a pronounced limp and a seriously deformed back/hip, so I suspect he had some skin in the artificial joint game.) He was fairly intense, and was one of those guys who always seemed on the edge of blowing his stack, though in fact he was always decent and patient with us students. That was one of my all-time favorite classes…

    The optimization program itself was called CADOP5; it was written in FORTRAN and it was a bear to use. (I remember him telling us, at the start of the course, that we needed to start working now on our optimization project, because it couldn’t be finished in a day, or a week, or even two weeks… I was probably the only one in the class to get an A on the project, and I wasn’t even able to find the optimum solution — though I was able to say why.) When I searched for CADOP5, I didn’t find much except this thesis at NJIT. The program given in that paper was a refinement called CADOP8; I might have met the student working on it at some point, but he had his PhD by the time I took the course with Dr Pappas.

    I really did like that class a lot, and its professor, and in fact tried to get Dr Pappas as my thesis advisor, but I got some behind-the-scenes stonewalling from one of his other students/assistants (it took naive me about a year to realize that maybe I’d been undermined), so did my thesis on linkages under Dr Sodhi, and the rest is history — or at least, water under the bridge.

    Back to the Python genetic algorithm: I downloaded it and wrote a little program to test it. It was amazingly easy to use, like all things Python, but it seemed pretty slow to me. (High computation cost? Slow convergence? No idea.) There are many other optimization critters out there in the Python ecosystem, I wonder how they would stack up against that one particular genetic algorithm module, or against those ancient CADOP programs for that matter.

    Anyway, something from the Wayback Machine: this and this are what I was writing fifteen years ago, maybe a month or so before I met Anne.


  • The Corliss Comes Alive

    Happy Pi Day! Here is that post about the steam engines.

    The National Museum of Industrial History has a few gigantic, spectacularly beautiful old restored steam engines, some of them even in operating order. I’ve seen the steam exhibit before, but I have been meaning to go back to the museum to see it again, because my friend Donna’s father just helped restore a new one they got. (George is a retired woodworker and very handy.)

    That new, and newly restored, addition to their collection is a Colt-Baxter “portable steam engine,” patented by a guy named Baxter and manufactured by Colt Firearms as a way to diversify after the Civil War. The Baxter was “portable” in the sense that it was only as big as a big barrel rather than building-sized; it ran at about 15 psi steam pressure and put out about 10 horsepower, was built to run a belt drive, and was ideal for powering small factories, machine shops etc — they sold maybe 300,000 of them over the years. (I learned all this at the Museum on Sunday, and on the Internet yesterday…)

    The museum had a demo day Sunday, where they would power up their Corliss engine — the biggest steam engine they have, and beautifully restored — using compressed air. I figured I’d kill three birds with one stone by riding the Iguana over for a test ride, watching the Corliss in action, and seeing the new Baxter engine on display.

    Here’s a video I made of the Corliss:

    At about 19 seconds into the video you can see what makes a Corliss engine a Corliss: the spider-web of levers running off a central rotating plate are what control the steam valves that feed the pistons. This engine was used to run a water pump; I’m pretty sure that the black part (the front) is the steam engine end, and the green part at the back is the water pump.

    And, here are a few photos I took of the Baxter:

    Colt-Baxter Steam Engine

    (Along the wall in the background, you can see some belt-driven machines on loan from the Smithsonian, drills and lathes and such, that the Baxter would have powered.) The Baxter had its own furnace/boiler built into the lower section, with the piston inside the top of the “barrel” and the bulk of the machinery on top.

    The museum had a few other exhibits, including a few small model engines running, as part of the demo, and one final surprise for me: the Baxter engine was operational! They didn’t have a fire running inside it, it was all compressed air like the Corliss, but here is a video of the operator starting it up:

    In all, a banner day!