• Category Archives tech talk
  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • Oh, No Reason…

    I’m not sure why I did this exactly, but the other day I decided to download a QR code generator onto my phone. I have no real need, but it looked like a fun thing to play with, so I made a few codes (my contact info, “Hello World!” etc), then I thought it would be pretty cool to read and write them from the laptop, so I downloaded a program called qrencode to write them, and one called zbar to read them, and I had a bunch of geeky fun using all my new toys.

    Then I got the idea: what if I could take a picture of a QR code, with datestamp and GPS metadata added? I could then extract the QR data, and the time and place it was gathered, like maybe something an inventory program would use. I downloaded another program called exiftools, and found how to get the date/time and location from the photos, but the final step, extracting the QR data from the photo of the QR code image, was a failure. I have no idea why yet.


  • Hidden Figures

    We saw it the other day, basically as soon as it was out in a nearby theater. We happened to go on a weekday matinée, which is what we usually do, but unlike other matinées the place was packed — it looks like we weren’t the only ones who wanted to see this movie. And it did not disappoint: this was one of the few times where the movie audience applauded at the end. My advice: go see it. (You’re welcome.)

    The story follows three black women who work as “human computers” for NASA in the early 1960’s. “Computer” was actually what they were called; it was a real but low-status job for low-status (female, black) math whizzes in the days before electronic computers, and there were rooms full of them, like steno pools, at NASA. This being Virginia in 1961, our three heroines were relegated even further into the segregated “colored computers” pool. So with the budding Civil Rights movement as backdrop — and this movie excelled at backdrops, with an awesome period score and loads of what looked at least like archival footage — these women broke through racist and misogynist barriers, and got John Glenn into orbit.

    And then, just as electronic computers started to threaten their human computing jobs, they figured out how to be the ones to do the necessary work of programming those computers. (It wasn’t in the movie, but programming back then — difficult, exacting, requiring daily brilliance just like now — was another low-status job for “girls.”)

    One thing caught me though, not in the story itself but in how the movie was put together. I remember reading once about how some movies were subjected to audience polling, and changes based on that polling, before final release — I wasn’t quite aghast, but it kind of irked me that this was done, and I started seeing what I thought was poll-driven editing everywhere in the  movies I watched, and I thought I spotted it here.

    There were two (three) parallel stories going on: one (two) involving lowly employee showing them how it’s done, and the other showing the futuristic but inert IBM that NASA purchased being brought to life. The stories were finally brought together, mostly by the  juxtaposition of the two “TRIUMPH! THE END” endings, but at one point there seemed to be an aborted attempt at a connection…

    The top NASA engineers are trying to figure out some orbital mechanics and realize that they need a different mathematical approach, and Katherine Johnson says “Euler’s Method!” Eureka! But then that’s it: other than a scene where she reads up on the method  in an old text, there’s no follow-up. The thing is though, Euler’s method is a numerical method, made up of many simple calculations instead of a few sophisticated ones, and it’s prohibitively impractical as a tool without the electronic computer. I can almost see the missing scenes, where Katherine’s superiors despair of getting the answer in time because there’s just too many calculations, just as Dorothy Vaughan got that old IBM up and running in time to save the day — oh what might have been! …but that’s getting nitpicky, me dreaming up extra scenes, just because I wanted the movie to go on and on.

    This movie was morally affirming — righteous even, and patriotic — without being preachy, pro-science without being hokey, and overall a pleasure to watch. Go see it, and see if you don’t applaud too at the end.

  • Administrative Editorializing

    “I woke the President to tell him we were under attack by the Russians!

    Do you know how stupid that makes me look?!!”

    War Games

    I moderate comments here: if you’ve never had a comment approved, all your comments go into a holding tank until I either approve or trash them, though once your first comment has been approved your subsequent comments are all automatically approved. It usually doesn’t matter much, since I don’t get many legitimate comments and have only one commenter, but that’s the way I like it because it blocks comment spam.

    The other thing about comments is that I get an email every time one is posted. This is on my “extra” email account, which doesn’t get much use, especially after I unsubscribed from a mailing list I was on. Then this afternoon my phone dinged a few times, and when I looked I had 22 messages, all from this site and saying I had comments in moderation…

    My site hadn’t gone viral, it was all just robo-spam: gibberish with a couple of websites thrown in, that kind of thing. I dealt with that set of comments by trashing them, and then a few hours later I got more, which I also dealt with. I noticed, though, that despite different names and email addresses, they were all coming from two internet addresses. I blacklisted those addresses, so now the comments go straight to trash, and I don’t get email notifications.

    I just checked the comment trash here, and it had a ton of spam comments. I guess I’ll have to check and empty the trash every so often until this entity gets tired of sending them, but as far as I’m concerned it’s problem solved.

    By the way, the offending internet addresses are assigned to a Russian ISP.

    UPDATE: The spam continued for about 12 more hours then stopped.

  • A Cone At Home

    I had a problem to solve at work last year, basically to make a cone out of bent tubes, to form a cone-shaped “throat opening” in a wall made of vertical tubes. The task needed a bit of iterative trial-and-error to solve for each tube, which quickly becomes tedious when there are maybe a dozen tubes that have to be looked at — half a day’s work — for any given throat configuration, and there were a bunch of configurations we wanted to explore.

    You can read about it here, but after that first day of tedium I decided to see if I could automate the process. I wrote a short C program, including set of vector functions and a root-finding function (using the Bisection Algorithm, which is supposedly slow but fast enough for my purpose — more important to me was that it’s pretty robust, and guaranteed to work in my situation), to find the necessary workpoints and design requirements for an individual tube in the cone. I then wrote another program to generate the input data for each individual tube, based on the tube, wall and cone parameters. I could give the “cone_maker” program the tube OD, bend radius and minimum allowed straight between bends (tube parameters), the number of tubes and tube spacing on the wall (wall parameters), and the cone inner and outer diameter (cone parameters), and pipe the results through my original “throat tube calculator” program, to get the data I needed. The programming took about two days, maybe a total of four actual hours of programming time, and it ran — flawlessly — in seconds.

    Unfortunately, to use the program I had to go through a whole rigmarole, running it on my SDF free shell account and accessing it on my phone via ssh, since we had no real resources for running or compiling programs at work. The process was faster, but still very tedious — you try typing dozens of numbers into and reading the results off a tiny phone screen — but it got the job done.

    The program did what it needed to, and it looked like I wouldn’t ever need to use it anymore, but I started thinking about program improvements to make the tube design process easier. You can read about these changes here, but what I decided to do was add new output options to the throat tube bend calculator: one option that produces AutoCAD commands to draw the “skeleton” of the tubes, and another to create a lisp file (AutoCAD uses lisp as its scripting language) to make a 3D model of the cone tubes. This took more work than it needed to because checking the results had to be done at work, while coding had to be done at home, but within days I had the program output running smoothly. I then armored the programs and turned them into a CGI script, and made a web page to access it.

    Throat Calculator Outputs in AutoCAD

    Here’s the calculator web page, and the results can be seen to the left. I had absolutely no use for the calculator anymore, but it sure was fun to play with.

    Fast forward to now, and I thought it would be fun to play with again — unfortunately, I don’t have AutoCAD at home, and am not likely to get it anytime soon, but I do have a program called FreeCAD. Now FreeCAD does not use the same things AutoCAD does, but it does have a built-in scripting language: Python.

    Python has been on my radar for a while, and with my recent QGIS forays (QGIS also uses Python as a scripting language) I’ve been motivated to learn a bit more about it. Then I happened to see my version of FreeCAD get auto-updated the other day, and thought it would be nice to play with, and maybe pick up on some Python on the way….

    bent tubes
    Python-generated quadrant of the cone.

    So I rewrote my cone maker & tube calculator programs in Python script. Much (but not all) of the vector stuff is available in a library, and so are root-finding algorithms — just for laughs I used Brent’s Algorithm, a faster version of Bisection — and Python code is naturally more compact-looking than C, so the final program looked really nice, and much shorter than my original C programs. In terms of running, there seemed to be a lag at first (probably importing all the libraries I called for), but the output just about spit itself out.

    Once I got the program to produce correct numerical output, I moved it into FreeCAD and started figuring out how to create the tubes. This took a bit of research, and a bit of trial and error, but the whole learning process took less than a day and then it was running beautifully — you can see the results to the right, and the full throat below.

    cone made of tubes
    An isometric view of the full cone.

  • Learning QGIS

    So what else have I been up to lately? I decided to look a bit more closely at QGIS, the open-source GIS program, and so I found a group of online courses on using it. They’re free, and you kind of get what you pay for here, but they’ve been an eye-opener into QGIS and its capabilities — it’s a much more powerful program than I realized, and with the ability to run R, GRASS and Python scripts, as well as automating tasks (and linking them together like unix pipelines), it’s got almost limitless expandability. I’m working through the third course (of five) right now, and when I’m done with these I may start looking into possibly going further.

  • A Grand Experiment

    I love my Garmin, but the  map it came with was horrible, so I replaced it with one from OpenStreetMap. (This is not news; I got the map years ago.) The process is tedious but pretty simple: there are sites you go to, and you pick what parts of the world you want a map of, then they do some data processing and email you to let you know when you can download your map file. The files are huge, like 3-4 GB for the one I got for North America, and they take a while to process and even longer to download. But once you have the file, you just put it on a micro-SD card, stick it in your GPS, and voilà — a much better map!

    Maybe it was the choice of map file I made back then, or maybe OpenStreetMap back then was less complete, but my map didn’t have many offroad trails. I didn’t feel the lack too sorely, since on most of my offroad rides I already know where I am, but after the last big ride — when I had become a bit lost — I looked at our path on the latest OpenStreetMap cycling map, and I saw all the trails through the strip mines — singletrack, jeep road and all. Boy, wouldn’t that have been nice to have on the ride! I also noticed that all the trails on Broad Mountain are now on the map, including the “secret singletrack.” I’ve done a couple of (road) rides recently, where I mapped out a course online and then downloaded it to the Garmin and used its routing features, “turn left onto Main Street in 100 yards” etc, to follow my course, and I thought that it would be a great thing to try routing with an offroad ride. The only thing I’d need would be routable trail maps…

    My understanding of the Garmin 810 is that multiple maps can be installed and enabled, and I’d been reading up on how to make the Garmin maps. (For years I thought it would be a cool project to make small custom maps of local trail systems, either standalone or as add-ons to a base map, but other than some re-purposing of GPX tracks I never really pursued it.) I didn’t feel like going through the process of downloading another huge (updated) map of North America from that map service again, but generating much smaller add-on maps myself, using OpenStreetMap data and the same software the original map service used, seemed to be fairly straightforward, and I could make a smaller updated file to add to my base map.

    So that’s exactly what I did: I downloaded the data for a region around Jim Thorpe and saved it on my machine, then ran a Java program called “mkgmap” to create the map file. Installed it on my Garmin, and voilà — the trails were there! I then created a course online, following some Broad Mountain trails I know well, and arranged to go riding with Rich B.

    Results were mixed. Our ride was great, but the downloaded route beeped an error message while loading and would not do any routing, though it would show the route on the map, and would indicate if we went off course. I got home and found that I’d compiled the map without routing capabilities, so I recompiled and reloaded my new map; it still awaits testing since I don’t get up to Jim Thorpe in a very regular basis.

    Meantime, I thought I’d make a similar map for the trails at Lake Nockamixon, since I did have immediate plans to ride there. I drew up a course to follow (which worked fine), and compiled a map of the Nockamixon area, but this new map would not display on, or even be recognized by, my GPS. I tried making a few other maps, but the only one that ever worked was the original Jim Thorpe one, and I have no idea why. I eventually got so frustrated that I went out and bought a new micro-SD card, and re-downloaded the map of North America, a process that took about six hours (though I wasn’t actually present for most of it).

    My next offroad ride will include a test of the trail routing capability of my new map. It better work.

    (Just as an aside: my resting heart rate this morning was 49 BPM.)

  • Compiz: Goodbye To All That Eye Candy

    A computer update: the file selection dialog boxes on my machine, as well as quite a few programs, rely on GTK+3 widgets, but my desktop is really MATE (using GTK+2), and the generic theme that the GTK+3 stuff gets rendered in was just plain ugly, so I added an upgrade to the Cinnamon  desktop, and have been playing with that for a bit. My thoughts so far:

    1. Cinnamon seems faster, and seems to also use less computing power.
    2. It (Cinnamon) seems less complete, and more buggy, than MATE.
    3. It has some clean looks, but it is really plain, and all my favorite little pieces of eye candy are gone.

    I tried going back to the MATE desktop, but I found that even though I enjoyed having my toys back, the ugly GTK+3 programs really were too much, and so I’m trying to get used to my new plain-Jane desktop. (I should look into installing a MATE theme that works for both GTK+2 and 3. Then I could go home again.)

    One motivation for changing desktops was that I was planning to use Eclipse as an IDE for Python, which meant that I had to start using Eclipse, one of the ugly programs under MATE. I also upgraded to the latest Eclipse, and installed Java 8, the latest Java runtime environment, and added the PyDev IDE plugin. OK, all of that went well, though I did things in a slightly nonstandard fashion, and I now have an up-to-date Eclipse IDE with Python, as well as C/C++ and Javascript installed.

    But in the meantime… The next time I fired up the Java Open Street Map editor (after installing the new Java), JOSM just puked a bunch of error messages. No idea what went wrong, but I was launching it with the old Java Runtime Environment, and I thought that maybe there was an interference with the new environment I installed. So, I tried launching it with the new JRE, and it came up just fine, along with a message that, at long last, it too had been upgraded to the new Java 8, purely by coincidence but at the same time as my own fiddling.

    Anyway, all is well again, if slightly dull, in my computer land.

  • One Day On The New Bike…

    …and I am incapacitated.

    I took the bike up to Sals for its maiden voyage, and I managed to catch a tree with the left handlebar — the handlebars are much wider than the ones on my Turner, or any of my other bikes — on a downhill no less, just after the 3 B’s climb, and it dumped me at speed into the rocks. I landed on my right knee, hard enough to see stars, and to literally bounce across the trail and roll down the hill. It took me 10 minutes to even get up, I was convinced I’d broken something, and I had to walk most of the way home. I spent most of the afternoon and evening with ice on my knee, and I will be doing the same today.

    Other than that it was an OK ride, not awesome but OK. The Santa Cruz rides quite differently than the Turner did, and there will be some things I will just have to get used to, and a few things I’ll need to do — suspension adjustments, seat height, possibly change the handlebar length (shorter) and the stem length (longer) — to dial in the ride. I would like for the bike to be a bit more responsive in turns, but that may come with time and those adjustments.

    There are three pieces of new technology, new to me anyway, on this new bike: tubeless tires, an adjustable seatpost, and 1×11 gearing. The tires are probably an improvement, but one — the absence of flats — that I might not really notice, and the seatpost is a cool gimmick so far, but it’ll be a while before it’s really incorporated into my riding; the new gearing is a bit more problematic. I went from 17 effective gears on the Turner’s 3X9 to just 11 here, and it seems like I have less of a high end, and less of a low end, as well as a less fine-grained set of gear choices. This may be the hardest thing to get used to, but there is apparently no going back: triple chainrings, and even doubles, are being phased out on mountain bikes, this is a weight savings for what could be a bigger and heavier bike, and I think the Santa Cruz has a lower bottom bracket, so a smaller, single chainring helps with ground clearance.

    Anyway, the bike seemed to perform well, especially on downhills, though the big crash wasn’t my only one yesterday, and though it seemed both twitchy (the short stem) and hard to turn (the long wheelbase) it did well enough at Sals. Unfortunately, it’ll be a while before I get to ride it again, and even worse, I’m going to have to bail on the Wilderness 101 this weekend.

    That’s right, no W101. We saw Renee last night and I had to give her the bad news. I felt like such a disappointment, but I won’t be walking much, much less riding, in the next week, and even if I could ride, my knee could never handle 100 miles the way it feels now. Timing is everything.

  • Live And Learn

    Posted on by Don

    I’ve had trouble recently with using my email here at donkelly.net: some — not all, but some — networks wouldn’t communicate with mine, emails couldn’t be exchanged, and looking into why that was so, at say, SDF.org I found that they couldn’t even resolve my domain. The domain always resolved on my home computer though, so some DNS was working somehow.

    But my laptop generally uses the DNS server on whatever wifi it’s connected to, and now, connecting here on Rainbow Lake, whatever DNS server they use wouldn’t resolve my domain — which I took to mean uh oh, there really is a problem with my setup and not just at SDF or whatever.

    I checked my DNS info using third party websites and found that here were some major discrepancies — there were four nameservers listed rather than two, and two of them didn’t work. Turns out the original ones had been retired (by my service provider) but my system hadn’t been updated, and the broken servers were the retired originals, which were the only ones listed in my site’s configuration — I have no idea where/how the correct nameservers got listed. I went in and removed the bad servers, added the good ones to my configuration, gave it a few hours, and now even the formerly broken emails seem to work.

    I could have, and should have, done something about this months ago.

  • Befuddle Oneself Methodically

    I’ve been thinking a lot about the economic impact of the towpath lately, and have been looking to the Great Allegheny Passage as a model, where many food and lodging places have popped up to cater to the cycle touring crowd.  I know that the D&L Corridor people are also looking at various businesses and how the towpaths might impact them, but I believe that they are looking at it from a county-wide perspective, when they should probably really be looking at impact within a few blocks of the path, and or at least within about a mile of the river — would you decide to take a fully loaded touring bike miles out of your way, and probably up some hill to get away from the river, just for say, lunch, if you didn’t absolutely have to? So, that got me thinking about the question: what restaurants, hotels, bike shops, and other amenities are actually within a mile of the relevant sections of the  Delaware and Lehigh Rivers?

    (This was a good first approximation, but it’s surely a naive way of looking at the problem, since there are many places within a mile of the river as the crow flies, that are not actually within a mile, or maybe even many miles, of anyplace accessible from the towpath — and places that will see towpath business will need to be located within a matter of blocks, not miles, from towpath access points. But I realized all that later as I thought more about the overall situation, and my first analysis of towpath business prospects was what I worked on first.)

    The way I looked at it, my original problem broke down into two parts. First, what is the region within one mile (or whatever distance) from the river, and second, what are the amenities within that region? The first part was fairly straightforward, but the second, which looked like it would involve some kind of Google Maps search — and eventually it did — turned out to be more complicated than I thought…

    Partial map of Lehigh County and River
    Nifty QGIS: Five minutes of work to get the buffer zone.

    I used QGIS to deal with the first part. I took as my reference some Pennsylvania aerial photos, plus a property map of Lehigh County, and created a new line vector (in a projection that uses feet as a unit of measure), following what looked like the middle of the Lehigh River from about Laurys Station to just past Bethlehem, and then I used the “Create Buffer” geoprocessing tool to create a vector polygon buffer region around that section of river, whose distance from my line vector was 5280 feet, in other words, one mile. That part worked great, but what to do with my buffer region?

    My first thought was to take the buffer vector and export it to a KML file, import that KML file into a custom Google Map (using Google’s “My Maps” personal map creation/editing feature), and then “do something” with it. That all worked great as well, up until the “do something” part — the KML file, and the personal map, were not much use when it came to customizing a map search.

    I did find online, however, that there were some things you could do with polygonal regions, and testing locations (such as the ones returned from search results) to see if they fell within those regions, using the Google Maps API. This added two new steps: first I had to re-export my buffer region, this time as a GeoJSON file because that was what the API would accept, and I also had to sign up for an API key from Google Maps. Both of these were also straightforward and easy to do.

    The final step was to put it all together: make a web page, and (with some javascript), load and draw the GeoJSON file, run a search (for restaurants, in my experimental code), and then find and display results that fell within my region. Code code code, give it a try… nothing. I was able to load the file and see my region, but no place results would be displayed.

    Turns out, there is more than one Polygon type in the API, and the one created by loading a GeoJSON file is different than the one you can test locations against; I would have to convert my polygon from one form to another. (This seemed to me a bit much, especially since I thought I should have been able to load the original KML file and it would “just work.” After all, isn’t KML a Google thing, and kind of a standard?) No matter, the conversion process from one polygon to the other looked as straightforward as every other step so far, so I just added it to the end of the task chain. Code code code, give it a try… nothing, and here is where it started to get really frustrating.

    I couldn’t for the life of me figure out what was going wrong, it looked like I did things exactly the way I was supposed to but my new, converted polygon could not be made, and it looked like the original polygon actually was empty, even though it was drawn on screen. I eventually used a callback routine from the GeoJSON loading function to get the polygon coordinates, and for some reason that worked.

    That gave me my clue: the “some reason” was that the callback was not executed until after the file was done loading, so the conversion routine had something — a non-empty original polygon — to work with, while in my original code the rest of the script wouldn’t wait for the file to finish loading before continuing, so there really was nothing to work with yet when I tried to do the conversion. That took three paragraphs to write, but more than a day to work out…

    I didn’t really like my solution: if you’re forced to use callbacks like that, you end up going down the rabbit hole, callback after callback after callback, just to get some semblance of sequential execution. (Meantime, I found that some methods did not suffer from these kinds of problems, they seemed to wait for the data to load before trying to work on it. Strangely enough, all the simple API examples I found at Google used these methods instead of the one I needed.) Eventually I set up a wrapper function to hide the messy details and just get me my goddamned polygon from the goddamned GeoJSON file.

    Anyway, here is my demo map:

    And here’s my script. Most of this is based on Google Maps API examples, but the function getBuffer() loads the data, and createBufferPolygon() is the wrapper that creates the polygon object:

      var myNewMap;        // the google map
      var myPlaceService;  // object for google places api
      var myBufferPoly;    // the polygon that holds the buffer region
      var myInfoWindow;    // info window for the selected place
      // the callback function from loading the API, where everything actually happens
      function initMap() {
        myNewMap = new google.maps.Map(document.getElementById('map'), {
          center: {lat: 40.672628, lng: -75.422778 },
          mapTypeId: google.maps.MapTypeId.TERRAIN,
          zoom: 11
        var bikeLayer = new google.maps.BicyclingLayer();
        myBufferPoly = createBufferPolygon(
        myPlaceService = new google.maps.places.PlacesService(myNewMap);
        myInfoWindow = new google.maps.InfoWindow();
      // this is the wrapper function, which calls the function that loads the GeoJSON file
      // after creating the polygon to hold the buffer region 
      function createBufferPolygon(url, featureName, map, isVisible, polyFillColor) {
        var bufPoly = new google.maps.Polygon({
          map: map,
          clickable: false,
          visible: isVisible,
          fillColor: polyFillColor
        getBuffer(url, featureName, bufPoly);
        return bufPoly;
      // this function loads a GeoJSON file containing a named polygon
      // then adds it to the given polygon object
      function getBuffer(url, featureName, poly) {
        var bufGeom;
        var bufferData = new google.maps.Data();
          {idPropertyName: 'name'},
          function(featarr) {
            bufGeom = bufferData.getFeatureById(featureName).getGeometry();
      // finds all restaurants within 15km of a certain location 
      function getSearch() {
        var request = {
          location: {lat: 40.703117, lng: -75.416561 },
          radius: 15000,
          keyword: 'restaurant'
        myPlaceService.radarSearch(request, displayResults);
        // displays search results that fall within the buffer region
        function displayResults(searchResults, searchStatus) {
          if (searchStatus !== google.maps.places.PlacesServiceStatus.OK) {
            console.error("getSearch error: " + searchStatus);
          for (var i=0, result; result=searchResults[i]; ++i) {
            if (google.maps.geometry.poly.containsLocation(
              result.geometry.location, myBufferPoly)) {
      // adds marker for selected places
      function addMarker(place) {
        var marker = new google.maps.Marker({
          map: myNewMap,
          position: place.geometry.location,
          icon: {
            url: 'http://maps.gstatic.com/mapfiles/circle.png',
            anchor: new google.maps.Point(10, 10),
            scaledSize: new google.maps.Size(10, 17)
        google.maps.event.addListener(marker, 'click', function() {
        myPlaceService.getDetails(place, function(result, status) {
          if (status !== google.maps.places.PlacesServiceStatus.OK) {
          myInfoWindow.open(map, marker);

    That’s a lot of work for something that solves the wrong problem! My next look at this will likely just involve finding the access points, and doing Google searches near each one — soooo much less elegant…