• Category Archives tech talk
  • Computers and programs, maps and GPS, anything to do with data big or small, as well as my take on the pieces of equipment I use in other hobbies — think bike components, camping gear etc.

  • Another Way To Look At It

    In my database of towpath-accessible amenities, my original definition of “accessible” was simply “within a half mile, by paths available to a cyclist, of an access point.” This seemed pretty reasonable — people don’t really want to travel more than a few hundred yards off the trail to get a bite or whatever, and beyond that point an amenity isn’t really part of the trail ecosystem anymore. This break-off point is more or less arbitrary, so I chose a half mile as a nice, round and fairly inclusive distance.

    As an example of what this might look like, here is a view of the Sand Island trailhead, with accessible amenities selected using this simple definition:

    map of bethlehem
    Amenities within a half mile of Sand Island

    You can see that there is not much available immediately near the trailhead, and then further north there’s a hotel (blue square) and a bunch of food/drink amenities (yellow circles) on Main Street’s “restaurant row.”

    I saw two issues with this. One is that people will probably be willing to go a bit further from the trail to get to lodging or a bike store, but these places are not shown, and the other issue is that the arbitrary break-off point comes in the middle of a dense clump of amenities — this is the case with “restaurant row” — and it seems silly to include one restaurant just under the half-mile cutoff while excluding the four next door, just beyond it.

    So (to address the second issue) I expanded my definition to include clusters: if there is a dense group of amenities, and at least one of the amenities in the group is within a half mile of the trailhead, all of them in the group are considered accessible.

    I also included any lodging or bike store within a full mile of the trailhead (to deal with the first issue), and now the map looks like this:

    map of bethlehem
    Amenities with a more expansive (ie “clustered”) definition of accessible.

    The map now includes the rest of the tourist/nightlife area near Broad and Main Streets, as well as several more hotels and two bike stores (the orange diamonds). This is the version of “accessible” my amenities map uses.

    But should I be even more expansive in my definition? What if someone wants to find a restaurant near their hotel, or near a bike shop, and the hotel or bike shop is one of the outliers, nowhere near the other “accessible amenities” even if there might be places nearby? (This actually happens in Bethlehem, where there is a separate business district on the south side of the river.) I decided to explore this possibility.

    My first approach to a more expansive definition of accessibility was to expand my definition of an accessible cluster: if a group of amenities contains anything considered accessible, like an amenity within a half mile of the trailhead, or a hotel or bike shop within a mile of the trailhead, then all amenities in the group are accessible. The new amenities can be seen in this map:

    map of bethlehem
    Sand Island amenities, including clusters near hotels and bike shops

    This was easy to implement, it just meant a tweak or two to the function I used for the original cluster definition. Now the Southside downtown is pretty well represented. By the way, here is the same map, but with clusters (accessible and inaccessible) shown:

    map of bethlehem
    Amenities near Sand Island, with cluster regions shown.

    The colored polygons in this map are the regions where clusters of amenities can be found. (Note the yellow polygon in the northeast corner. That represents a cluster where no amenity met any of my criteria, and no amenities are shown.)

    I worked out this definition of accessibility about the same time as the original clustered version, but didn’t use this definition for my map because it seemed a bit too expansive, with these second-order amenities — accessible not to the trailhead per se, but to other places that someone may want to visit — making the whole map too busy without adding much more value. After all, if someone wants dinner recommendations near their hotel, they can always ask at the front desk…

    But lately I’ve been thinking about a different approach to expanding my definition: what about the routes from the trailhead to the hotel (or bike shop), would it be useful to show the amenities along the way? Here is a map, just of the amenities that are within 50 yards of the shortest path to the accessible hotels and bike shops:

    map of bethlehem
    Amenities along the routes to hotels and bike shops.

    This seems to strike a happy medium, inclusive but not too inclusive. Here are those amenities along the hotel/bike shop routes, with the more expansive version of the clustered amenities superimposed:

    map of bethlehem
    Amenities near Sand Island, including those in clusters near, or along routes to, lodging and bike shops

    I kind of like this approach, though I wonder if it’s more a CYA reflex: I don’t want hungry people to pass restaurants on the way to the bike shop, and not see them on my map. After all, this is still a second-order set of amenities, and even if it’s not quite as busy as my first attempt at a more expansive definition, there is a lot of overlap. I’ll be thinking about this a bit more…


  • Infrastructure Fun

    I got in a few rides these past few weeks, and some good cello time too, but my major focus has been on “infrastructure” projects:

    Bike

    The Santa Cruz, after four years of that “new bike feeling,” is starting to show some signs of age. Nothing bad, just things like shifting problems in the highest gears, so I might need new cables and maybe housing, and some trouble with the tire valves: I’ve got a slow leak in the rear tire caused by a torn o-ring, and a gummed up valve up front.

    For the tires I got a “valve repair kit” from Saucon Valley Bikes. The tubeless tire valves are pretty easy to take apart and work with, so I was able to replace the rear o-ring — I can’t be sure if it worked perfectly, but it’s working enough for now — and have new valve innards on deck if the front tire becomes too annoying. The shifting seems sort of OK for the moment after I did some serious derailleur cleaning, but I can tell I’ll have to deal with those cables sooner rather than later.

    Meantime, I noticed a slight creak coming from the bottom bracket…

    SSL

    For my website I’ve been using an SSL/TLS certificate from Let’s Encrypt, which I obtained using SSLForFree, since Let’s Encrypt is pretty difficult on its own. These certificates need to be renewed every 90 days, but when I went to do it the next-to-last time, I found that SSLForFree had been bought out by ZeroSSL, who use their own certificates and who intend to charge for anything beyond a limited number of free ones. I used them that time, but spent the next three months looking into a better option.

    The ZeroSSL certificate expired a few days ago, but I had already replaced it with one from Let’s Encrypt, using a rather laborious process on yet another website. It’s very doable, but I think I’ll continue looking for a better method.

    Towpath Amenities

    This is a bit of old news, but I’ve added the amenities and access points along the towpath between New Hope and Morrisville. I have about 10 miles left to add, the section from Morrisville to Bristol, and I have all access points and amenities I could find added to my database. All that’s left is to ground-truth some of the info, then I can update the map. This last addition will make the map complete, but that won’t make the job done — this job will never be done

    I started thinking about my method of routing the other day: the routine finds the point on the road network closest to my access point (the start) and the point on the network closest to my amenity (the endpoint), then finds the shortest path through the network between start and end points. But what if the start and end points on the network are not particularly close to their respective access or amenity points?

    I originally assumed that this would not be an issue: access points were basically intersections of the D&L with the road network, and almost all amenities should be very near some road or path that customers use to get there. Then I figured out a way to check…

    Most amenities were within about 25 yards of their route’s endpoint, the distance being mostly open space like a parking lot or driveway. I figured that this was acceptable, but I also found a few amenities that were more than that distance, between say 25 and 50 yards from their endpoints. Again they were on the far sides of parking lots and such from the ends of their routes, but these distances seemed a bit too large to leave be, so I added service lanes and driveways as necessary — I’m not sure why these weren’t already a part of the network, but they are there now; I updated the routes to the offending amenities and all was well.

    There was a third group of amenities that I found, and these were the ones I had been worrying about: the ones where the database has a route, but in real life the route’s endpoint is nowhere near the amenity, and maybe the amenity isn’t even accessible from the endpoint. (One example could be a store along a roadway I’d deliberately excluded from the route network, such as a fast food place along a highway. The routing program would find a path to the closest point still on the allowed roads, and leave the cyclist to connect the endpoint and the amenity “as the crow flies,” crossing freeways or God-know-what, and I’m back to square one.)

    Luckily, I only found a few of these, and they all were total outliers: places that were in the database, but were too distant and isolated to be considered “accessible.” For now I’m leaving them in the database, but I guess I’ll eventually have to remove them. I’ll have to look more carefully at the relationship between new amenities and the road network in the future if I add any more, to make sure they actually connect.

    Network (the other kind)

    One last piece of infrastructure activity: we are switching our internet provider, from DSL on Verizon to RCN cable. I bought a cable modem and a wifi router, and called RCN the other day; the cable installers should be here this afternoon.

    I got us the slowest package, 10 Mbps, which is about four times faster than what we have now and costs about $20/mo less, before even considering the cost of the landline we’ll be abandoning when we get rid of Verizon. (If we need it we can upgrade our package, but we’ve been making do with DSL for so long that 10 Mbps will probably seem blazing fast.)


  • Now What?

    One of the pedals on my road bike has developed a squeaky bearing lately, and I thought that maybe it’s time for a new pair. (I’ve used the same clipless pedal system — Speedplay Frogs — for more than 20 years. With pedals on multiple bikes and cleats on my bike shoes, it’s a fairly big investment in the one technology.) I went online to order the new pedals, and found that they have become extremely scarce — like nonexistent, discontinued scarce. Turns out that Speedplay was bought by Wahoo, and they decided to shut Speedplay down while they “reconsider the product line” or whatever they might call it. WTF?

    My immediate options are to see if I can find a new pair on eBay or whatever (no luck yet), or to replace the bearing (which also requires an eBay purchase, but spare parts seem plentiful so far), or just keep re-packing the pedal with grease and hoping for the best — that’s what I did this afternoon. I guess I’ll eventually have to completely replace the Frogs with some new system, and rather sooner than later. Three sets of pedals and two sets of cleats — it’ll be a substantial chunk of change, but I’m not even sure what that replacement system will be yet. It’s a total shame, really, the Frogs are great pedals.


  • What Is Your Quest?

    Posted on by Don

    I’ve been moving forward with the additional D&L access and amenities points for my project, but the trail sections south of Riegelsville are terra incognita, especially when it comes to trail access, so I relied on GIS to find access points: I split the road network into “trail” and “not trail” sections, and intersection points (that aren’t at bridges) made for pretty good access candidates; some closer map inspections verified a few obvious trailheads, and weeded out some things like private drives. A lot still needed to be verified via “ground truthing” though, and so the other day I went out for a ride, starting from Riegelsville, south along the towpath to Tinicum Park.

    I had my candidate points loaded in my GPS so I could see on the map when I came upon one; I could add locations I’d missed, and delete false positives as I spotted them, and by the time I was done I should have a pretty good idea of how to get on and off the trail. This method worked really well, and the only real problems were judgement calls at what seemed like private access points. (Things are a little different in Bucks County, there are some wealthy homes between the trail and the river, with their own driveways and footbridges, and while some crossings are obviously marked “Private – No Trespassing,” others were maintained, and painted, as if they were park property.) Judgement calls, and I think I made the right calls, but for the most part it didn’t matter — all these these access points were too far from any amenities to be useful.

    It was easy and pleasant work, and I took pictures on the way back:

    Total distance, out and back, was about 24 miles, and the ride took about two and a half hours. I have the new access points and amenities incorporated into my map.


  • Yet Another Half-Finished Magnum Opus

    Posted on by Don

    I put the D&L trail data online, formatted as a map/website that I hope could be useful to someone, as soon as I fix a few tweaks — and add the missing portion of trail. Meantime,

    Click Here

    Enjoy! I’ll soon be adding some more useful UI parts soon, and eventually I’ll get to the trail stuff south of Riegelsville.


  • Trail Project Update

    Posted on by Don

    I got those new amenities into my D&L trail amenities database — it was a piece of cake, once the data was cleaned up. The whole process went smoothly, even the main database report (done through Jaspersoft Studio, which can be a pain in the neck to work with) digested the new information without any glitches.

    But I couldn’t leave well enough alone after that: I decided to add the amenities between Allentown and Northampton where the trail is incomplete. My original feeling was that the trail would be too vague through here — where are the “access points,” the intersections between the trail and the wider world’s road network, if you’re riding on the roads and there is no specific trail? But I’ve been riding this section a bit more lately, since it’s not been inundated with users like other sections, and discovered that much of it is only “unfinished” in the sense that it’s not up to the specifications of the rest of the D&L; it’s perfectly rideable on a mountain bike, and actually more fun than the more polished sections. If you don’t mind riding the rougher stuff there’s not much trail missing, and the remaining road portions are remote enough that there’s no need to worry about nearby amenities.

    So, I repeated the process for the incomplete section: identify trail access points, import amenities from OpenStreetMap (and then clean them up, by far the most laborious part) and finally tie access points to amenities using that routing distance matrix script. Again, it worked like a charm.

    …and then I started thinking about what I might want for output. That Jasper Studios report is nice, but the current output is a PDF — ugh, not very net-friendly — and I thought it might be nicer to get a more straightforwardly data-oriented output, something I can massage and format as necessary in a browser, something like JSON. Unfortunately, though Jaspersoft can do JSON output, I couldn’t quite figure how to get it to do what I wanted. Postgresql, the database I’m using, has JSON capabilities of its own, and final-product-straight-from-the-database seems like a better approach, but I didn’t know much else about those capabilities, so I sat down over the past few days and doped it out. The learning curve was pretty steep, more like a brick wall, and my code is pretty convoluted but I did get it to work. Of course, this success is just a lead-in to another escalation: if I want a web page I now have to code out the rest of the stack.


  • Tired of the Flats

    Posted on by Don

    So I got the Iguana’s rear flat fixed pretty quickly, it seemed like probably just a puncture or pinch from the towpath. I also took the time to do some deferred maintenance; now the bike is looking good and performing well.

    I then took off the Cannondale’s front wheel, removed the tire, and found what I suspected but couldn’t originally spot: there was a tear in the sidewall. Luckily I had a spare tire waiting, so I put that on with the new tube, and — BANG! That tube popped as well. I was done for the day…

    I looked at it a few days later and the most likely cause seemed to be “operator error” when installing the tube, no other problems or defects were visible so I replaced the tube and gingerly pumped the tire, with many stops for bead inspections etc… everything was fine. I let it sit for a few days, but heard no sudden gunshots in the basement, so I took the bike for a short ride Monday. All is good.


  • More Fun with Routing

    I’m not sure why I did it, but I installed PHP and Apache on my new computer, then moved a bunch of my “internal website” stuff over from storage. Everything seemed to work pretty well, so I tried the commuter routing program — I got errors, natch.

    I looked at the error messages and realized that the pgRouting routines had changed, so my database routing functions were out of date — that led to me discover that even the newer version of PgAdmin3 doesn’t work well with my newer Postgresql version, especially when it comes to functions. So, I installed phpPgAdmin — which was also borked, and in the same way, but I was able to fix the source code. Even working properly it couldn’t do what I needed though, which was to modify my old function. I tried writing a new function through phpPgAdmin, which was extremely laborious, and basically re-wrote the original, broken function, so now I had two useless functions that I couldn’t modify. Ugggh, time for bed.

    I woke up this morning and got it done old-school, writing a SQL script to define the function and running that from the command line. Presto, now I have a working function, and a working commuter routing program. Bonus: the new version of pgRouting is much faster (though that could be the new computer), and some routing errors are now fixed. Wish you could see it!


  • Fun With Alice And Bob

    I moved my SSH and PGP certificates to my new machine, and setting them up meant visiting a few cryptographic sites (to remember how to do it, and also just because). That means I spent a lot of time in that world, where “Alice” and “Bob” (and so on, alphabetically) are the main actors in usage scenarios: Alice sends a message to Bob, but maybe Clara intercepted it, how does Bob know the message is secure? Well, Alice encrypts her message and signs it with… Tedious, tedious, but now it’s done, and I can securely access my online accounts as well as send/receive encrypted email, which is sort of like being the only person with a phone — who can you call? — but still: I once thought it was cool, and now I have it back.

    Auld Acquaintance

    This past weekend was sort of the end of the holiday season for us: we saw friends we’d missed during the holidays. Friday night we did some bar-hopping with Doug & Lori, then Saturday was a “Festivus party” at Eric & Kris’s place (where we saw D&L again) lots of catching up and the usual merriment, it was a really nice night.

    Movie Night

    We saw “Uncut Gems” the other night. I don’t recommend it: the acting and the cinematography were all good, but the characters (especially the protagonist) were annoying and unsympathetic, and the plot, as Anne put it, was “a 2-hour panic attack.” Ugh.


  • Not Quite The Borg

    One more piece of software I added: Jaspersoft Studios, a program to design and generate reports from various data sources. I use this because — unlike other report programs I have tried — it works well with PostgreSQL, the database I like to use, but it can be a bear to use since it has a lot of moving parts.

    There are two ways to use Jasper Reports: one is as a standalone program, and the other is as a plug-in to the Eclipse IDE, which I happen to like using, and which I use nowadays for almost all my coding. I downloaded the standalone program first, but it looked so much like an Eclipse product it reminded me that there was a plug-in — this would be just one more thing that I could run from my single IDE to rule them all. So, I installed the plugin.

    What I got, plug-in and standalone package, was version 6.10, and it really looked beautiful. Unfortunately, as soon as I started to use it, I found that it was too broken to use… Jasper Reports is mostly like a graphical page layout program, where text and other elements can be moved and resized on the page, but the elements here wouldn’t move or resize. Which is bad.

    It took me a day to find information on the problem. I never did find a solution but I did find a workaround: just use an earlier version. That’s what I did, installing standalone version 6.8; Eclipse will have to wait until I learn how to add older plug-ins.

    Anyway, the program works fine now, and still looks beautiful. I used it today to make a table of bike rides from my database, natch.

    Reading

    I just got the latest Jeff VanDermeer novel, Dead Astronauts. This is set in the same story-universe as his Borne and A Strange Bird, and some of his short stories, but it is supposedly far more complex and experimental. More on this after I get through the first page…