• Category Archives tech talk
  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • An Addition To The Website

    A New Page!

    I just added a page of links to some of my Sals maps, you can also find it in my “other projects” menu. I’ll be improving and adding to it as time goes on, but there it is for now.

    I was hiking at Sals the other day and bumped into Eric riding with a few others; they asked me what I was doing and I told them I was tracking down the last of the trail marker posts, to add to my map.

    Eric was like “uh, what map?” and I said I’d been keeping a map of Sals for decades, so he of course asked me where he could find it — and that was the motivation for starting my new page.


  • Improved Wordle Cheating

    I use a script to help me with Wordle — creating the script, and watching it work (rather than the actual play itself), are what I like most about playing Wordle.

    My script works, in conjunction with a text file of the most common five-letter words, by counting the most common letters used in whatever subset of words are valid at any point in the game — part of the script runs the text file through a regular expression filter to get that valid word list — then rating the words by how many high-count letters they contain. This especially is a good tool early in the game, since it helps to either confirm or eliminate the most obvious letter choices right off the bat, but it has diminishing returns with later choices.

    I modified the script to similarly score multiple-letter combinations, but the results were mixed. Past a certain point, this approach really doesn’t do much, and adding letters didn’t change that. My average score hovered at basically just below four guesses per game…

    The “wordle bot” assistant has its own way to assess words, based on a “divide and conquer” approach: the results of each word selection basically split the potential words into several groups, where all the words in each group would have produced the same result if they were the answer. (I call the groups “buckets,” and the approach the “bucket method.”) The more buckets a word choice can split the potential words into, and the fewer words are in each bucket, the better that word is for zeroing in on the correct answer.

    I decided to try using this approach, writing a python script to do the work. It really was pretty easy, surprisingly so, and it worked really well so I added some sorting to it and incorporated it as an option into my original helper script.

    The results, again, have been mixed. I think my average score is now closer to 3 than 4, but I’ve also had a whole lot more fives (and some more twos) so the variance seems to have grown. My “skill” score went up, but my “luck” score dropped… Part of this (especially the “luck” part) is because I’m not selecting words as randomly, or weighting the odds like my old method did, so I now start with my go-to choice of “SLATE” (always, for luck), then use my original wordle-helper method to find the next word, and from there I use the new bucket method. Better, but still not great.

    It turns out, the new method is a bit more sensitive than the old one to unusable words in the word list. Wordle has a fairly large set of acceptable words (I think I saw somewhere that it was about 3000), but only a subset (say 2000?) can be winners, and none of them are plural (no ending in “S”), past tense verbs (mostly no ending in “ED”) and definitely no inappropriate words. There are may be other deal-breakers too, but that’s a pretty good set of exceptions. Meanwhile my potential word list, which comes in at over 5000 words, contains all of those and more.

    So I’ve been curating my list. I added some regular expression filters to avoid words ending in “S” or “ED,” and marked as many inappropriate words as I could find so they would not get considered. My results are now much better, though there are still a few words that my list considers common but the NY Times does not.

    Today’s score was a five, but I just had a long run of threes.

    About my choice of “SLATE”

    Even before I first wrote my wordle helper, I decided that “SLATE” was the best opening word, since it had what I thought were the most common letters, and the best combinations, to start the game. I’ve been using it ever since, and at some point I felt vindicated when the wordle-bot started using it as its own best starting word. Everyone else started using it too, but then the bot began to start with “PLATE” and my word became far less popular…

    Well, I was playing with my new script, running it for a starting word (that is, with no restrictions except plurals, past tense and cusses), and the best choice by far was “SLATE.” I feel vindicated again, though I don’t yet know why “PLATE” is the current opener of choice for the bot.


  • The End Of The Trail

    I haven’t been diligent about it, exactly, but I have been progressing on my Native Paths project, slowly but surely, and just yesterday I finished — I digitized my last path.

    (This was part two of my project, the second pass through the book. I have pretty much followed through on my workflow: for each path I’d photograph the map in the book, clean the photo using GIMP, georeference it in QGIS, and finally digitize the foot paths (and any native villages or other notable places I’d find) by tracing them from the map. I’d also check the description for any cross-references between paths, which I’d also add to the database.)

    A friend asked the other day what I would do with the project, as in how I would present it to the world, and I really didn’t have an answer. My original motivation (at least, as I remember it now) was not to produce something for other people to see, but to gain an understanding of the book Indian Paths of Pennsylvania for myself. I guess it’s time to take stock of what I’ve learned on this journey…

    I think I accomplished what I set out to do, which was to gain a better understanding of Wallace’s book, its scope and its limitations; I also have a better understanding of the world the Pennsylvania natives lived in. And in the end, I also improved my GIMP and QGIS skills — I got a lot of practice cleaning, georeferencing and tracing those trail map photos!

    I still have a few GIS projects I can play with — I can use the native paths project to plan some bike tours, and I also have a few “ground-truthing” things I want to do with my Sals maps — but the truth is, I have reached the end of a long, engaging, and sometimes onerous process, and I am feeling a bit of post-accomplishment blues. Oh well, something will come along, GIS or something else.


  • Re-Re-Re-Litigating

    I saw recently that FreeCAD, the engineering program I played with once or twice in the past (it wasn’t ready for prime time), well it finally just reached its first official release version. I thought I’d give the new version a try, so I installed it (using Flatpak — probably a mistake, but it was the easiest choice in the moment) and got it running pretty quickly. The program looks the same, maybe a bit more polished, but it definitely works much better. That got me thinking…

    There was a work project I automated years ago, then extended the automation; and then (after I retired) I decided to re-do the whole thing using FreeCAD and Python. Looking back, the actual project was a little bit boring, but playing with the programming was fun, and so I found my old programs, re-worked them a bit, and it worked great.

    I won’t bore you with new pictures.

    One difficulty I did run across was that FlatPak builds apps isolated in their own little sandboxes, with their own version of everything they might need in the sandbox with them, and the version of Python shipped with FreeCAD did not have some of the library moduless my original script needed. And I couldn’t for the life of me figure out how to install a library module in the sandboxed version of Python.

    All I needed from the library module was a root-finding function, and in the end it was actually easier to just write my own function than continue messing with the libraries. Strangely enough, my script now runs slightly faster despite my function being less efficient (bisection in Python vs Brent’s Method in C++ or whatever), probably because I got rid of the of the slowdown of having to load that library. Once again, for small problems, small brute-force solutions have their (small) advantages.

    (I don’t remember where I got the idea, all those years ago — I may have pulled it out of my ass made it up myself — but back in school I always thought that the magic number was 3000: when dealing with less than 3000 items, just use the brute-force approach.)

    I also went looking for some of the models I made in my last go-around with FreeCAD. I found a few and moved them from my old laptop to my new one — backing up the old laptop along the way, I don’t think it’s long for this world anymore — but the one I wanted to find, a model of a forged tee I made, was nowhere to be found. Oh well, I guess I can re-litigate that one too, if I really want.


  • Back To That Old New Thing

    I’m slowly digging out, getting back into some of my usual shenanigans: I went for a towpath ride today (and one several days ago), and I’m about to go upstairs to play my cello — first time in a month that I even took it out of the case…

    I also found another computer chew-toy, not so much a new thing as a return to something I was playing with a few years ago: I’m looking at pedestrian crash data using R. This time around, I have my data in a geopackage, and am using a different library to access it, which makes the whole process easier; I’m also limiting myself (mostly) to location data, and keeping the geographic scope limited to Bethlehem, though I am looking at a wider date range: 2013-2023. So far I’m having fun, but who knows when I’ll lose interest?

    We have Iris tomorrow, and Wednesday we’re visiting my Mom — I stop in to help her out with stuff maybe once a week. We’re also doing some backyard cleanup here, in preparation for the kayak storage racks Anne is building. Life goes on.


  • Some Bike Love

    I had a slow leak on the Santa Cruz’s rear tire, which turned out to be a stuck valve and relatively easy to fix (disassembled and cleaned it), but then the next day the front brake felt super soft. I got new brake pads, replaced the front brake — fixed! I did the back brake as well the next day, and I also finally got the shifting dialed back in. It feels like a new bike…

    I took it out on the towpath Tuesday after the front brake fix, then again on Wednesday with the rear brake and shifting done as well. Both times were awesome, but on Wednesday I took the little alternate “fisherman’s trail” in Freemansburg. Big mistake!

    The whole trail was completely overgrown with downed trees, sticker bushes and nettles. I fought my way through — it took forever, I had to shorten my ride I took so long — and when I was back on the towpath I got a good look at my arms, which were completely covered in blood. Yikes! It was from the stickers, and it was really just a bit of blood from the tiny sticker scratches, mixed with sweat and looking worse than it really was — the nettles were far more annoying to be honest. But it sure looked dire; I took pictures but I won’t post them here.

    Post-ride shower, and my arms looked like it never happened. Bike is still great.


  • Fun With Bash

    I got to play with shell scripts a few times over the past few weeks, thought I’d talk about them:

    Wordle Helper

    I have a script I use to help me make my guesses when I play Wordle; it basically generates a list of available words based on my results, analyzes these available words (by checking the overall letter frequency, then scoring the words based on which letters they contain), and returns the word list with their scores, sorted by word score. This is especially helpful as a process of elimination, and most especially in the first rounds — I can confirm or eliminate the most common letters and quickly narrow down my choices (well, usually).

    I decided at a certain point that multi-letter combinations might be even more useful, so I wrote another script to score words based on the frequency of two-letter combinations. This worked well enough that I wrote a three-letter scoring script, and then realized that I should just write something generic that would score based on however many letters I would want in my combinations.

    For some reason I brought my laptop on vacation, and when things got quiet I broke it out and wrote the multiple-letter scoring script — it works like a charm. What I found though, is that there are diminishing returns for the letter combinations: there is no real advantage to use more than two letters. Still, this is a script that I use every day.

    Getting SSL

    Every 90 days I have to get a new SSL certificate for this website, so I can use the more secure “https” rather than plain “http.” This was once a pretty easy process, until my (free) certificate-generating organization got taken over by someone else. I found a much more onerous (but still free) web-based method and have used it for years, but I knew there had to be a better way.

    I eventually found something called getssl, a bash script that automates the process used by that onerous web-based approach. Ufortunately it’s meant to be run on the computer where the website resides, rather than on my local machine which is what I preferred, so I never really messed with it… Then one day I just set it up and ran it, and it basically did 99% of the work, generating the certificates and storing them on my laptop. All I had to do myself was install them manually, which was pretty easy. Awesome! The whole process went from an hour to about a minute, and I’ve been using getssl for almost a year now.

    But it still bothered me that I couldn’t get the whole process automated, and then last week I found an extra feature in getssl that would get me that final one percent; the only problem was that the extra feature would need to be modified to run on my machine rather than remotely. (This modification turned out to be fairly straightforward, which was a bit of a surprise actually.) I managed to make the code change just a few days ago — again, while out of town with my laptop — and my current certificate was installed, fully automatically, using the modified feature.

    Neither of these little projects were hard, but it’s so seldom that I do anything like this anymore that I was pretty proud of my accomplishments.


  • Cancel The Exorcist

    Posted on by Don

    I’ve had a creaking near my mountain bike’s bottom bracket for a few weeks now. I was pretty sure it was either the crank or the bottom bracket itself (rather than, say, the suspension pivots), since it seemed to happen with every pedal stroke, as in an unbalanced load on the BB, rather than with the bike suspension flexing.

    I tightened the cranks — no luck, so i took them off, lubed them and reinstalled them. Still no luck, so I took them off to get to the bottom bracket, cleaned and lubed the threads and reinstalled them, and still had the squeak. Maybe my bike was possessed?

    I thought maybe I didn’t do the job well enough, so I took the cranks and BB off again, and this time I also took the rear wheel off to access the suspension grease port and pumped it full of grease. I put everything back together, and this time the creaking was gone. No idea what actually worked, but the pivots really did need the grease…

    Of course, the bikes never stay perfect for long: the MTB shifting is now a bit funky, and the touring bike had a flat tire when I checked on it. It’s always something.

    Saturday was a ride at Nockamixon with Tim C, someone I know from the Museum, and Sunday was a ride up the D&L to a coffee shop in Weissport. Hot rides, both days!


  • New Toys

    Posted on by Don

    Whelp, I’m now a kayaker…

    Anne and I planned to get kayaks as our Christmas presents to each other, and were doing our research when her brother & sister-in-law gave us two of theirs as a gift. We tried to pay them, but they refused — I think we did our part to help them clean their garage out.

    We got life jackets and paddles from them too, but we still needed some way to transport the kayaks. We did a little online searching, found some used roof racks for pretty cheap, and installed them just the other day.

    Today we put the kayaks on the roof and took them to Mauch Chunk Lake for their “New (To Us) Maiden Voyage.” Driving with them on the roof was a bit unnerving, but they did just fine. We got to the lake, got them in the water, and despite the windiness of the day, we had a great time with our new toys.

    So now we have the boats, and (some of) the infrastructure, and we’ve christened them — we are kayak people now!


  • The Things We Carry

    Posted on by Don

    Anne and I just did a section hike on the Appalachian Trail, walking the New Jersey portion with several friends. More (probably much more) on this later, but one of the things that struck me was how meditative just hiking along can be — songs, other earworms, and various musings would take up a lot of the mental landscape.

    I’d built a QGIS project before our trip, with info about our section of trail, and put it on my phone. It had some distances in miles and others in kilometers, and I was mentally converting back and forth when it struck me: a kilometer is about 0.6 miles, while a mile is about 1.6 kilometers. In other words, their reciprocals differ by one. Is that true, and if not, what number would it be true for? I worked out the quadratic equation:

        \[x^2 - x -1 = 0\]

    (hmmm, seems familiar)… and I found the answer to be:

        \[1 + \sqrt{5}\over2\]

    otherwise known as φ (phi), the golden ratio. Interesting, and a fine bit of mental bubblegum to chew on as I walked along…

    When I got home, I broke my internet fast with some Reddit, and the first thing I see there is a photo of a tee shirt, with the golden ratio written on it — meaning, in this case, the “most irrational” number, as in “I am most irrational.” Anyway, just a coincidence, but a strange and striking one.

    Click To See The Shirt

    (You can see the shirt by clicking the image.)

    The other thing I carried with me in my head? “The Alley Cat,” a perfect earworm for my hiking rhythm, though I noticed that I changed the tempo as the terrain changed.