I'm currently working on building a local delivery service using electric cargo bikes in NYC: https://hudsonshipping.co. We are planning to launch our first pilot in early 2026 with our first customers in Brooklyn.
We've built all of the tech in-house to manage the fleet, deliveries and optimize our routes. If you know of anyone that would like to be a part of the pilot program, feel free to reach out to me!
If a ping to a specific IP times out, I wouldn't say the IP is blocked. It could be that ICMP specifically is blocked, following some network rules on the firewall. This is pretty common in entreprise networks to not allow endpoint discovery. I could be missing something and happy to be corrected here, but I was surprised to read that.
I find it's important to remember, too, that a failed PING tells you nothing other than your echo request did not receive a response. If the remote host received your request, and if it responded, are both things a failed PING can't tell you, because both of those things could be true but you still end up with a failed PING.
I've seen technicians get tripped up in troubleshooting thinking that a failed PING tells them more than it does. When the possibility of asymmetric return paths is involved it's always important to remember how little a failed PING actually tells you.
And that can be a lot more subtle than you might think. I've had a persistent very hard to debug false alarm triggered on pings sometimes not making it and most of the time they did. But very rarely that would happen three times in a row and that was the threshold for raising an alarm. We spent days on this. Finally, the root cause was tracked down to a BNC 'T' connector at the back of a media adapter that filtered out the header of some percentage of ICMP packets. It is one of the weirdest IT problems I've ever encountered and it makes me wonder how much of what we rely on is actually marginal.
I work at a company that invented an internal syntax to compile into C++ code, that still relies on c-shell and conventions taken when OS/2 was in use there, and with a web of Jenkins instances and homemade wrappers and DBs to build that stuff.
I can safely say that title exists already. And I value my current experience as a humbling example of what is to come as software becomes an older industry, and not just a world of startups and their freshest languages/frameworks/tools.
I thought that too up until this GenAI moment, and now I wonder if needing to be an archaeologist will be so valuable if one can get your needs met by a quickly GenAI-written script/program.
> I thought that too up until this GenAI moment, and now I wonder if needing to be an archaeologist will be so valuable if one can get your needs met by a quickly GenAI-written script/program.
I never have actually read those books (though I read some summaries about them, interesting concepts). My understanding is the "programmer-archeologists" basically had an archive of massive quantities of very high-quality software that did pretty much anything you'd want software to do. So it made more sense to find the software you need and glue it together than write from scratch.
And given GenAI doesn't write high quality software (at least not yet, and hopefully never), I don't think that "GenAI-written script/program" would be a good replacement (though an AI archeologist might make more sense, with such an archive).
The world in question is ours but later, with direct lineage from Unix systems indicated. So I see these archaeologists as a glorified priesthood of shell scripters, grep still having bugs, and the glue being programs themselves. Not too different than many roles today.
Beyond that, it is an odd hope on your part for GenAI to never be able to write high quality software.
Zooming out, my bigger point was that this was a sci-fi book written by the person who coined the term and concept ‘Singularity’ and the series includes a malevolent murderous sentient AI virus (IIRC) and it included some reference to how programming was accomplished and yet still, given all that, there was no anticipation of even our nascent current GenAI coding capabilities.
I've yet to have my needs met by a GenAI-written script/program. Archaeologists tend to be a lot more precise in their statements, especially about what is speculation and what is not.
I mean, if you're willing to accept AI slop, that's fine. But if you're willing to accept AI slop, you'd probably be willing to accept human slop (at least if it claims to be AI) too, and then the job gets a lot easier.
We’re talking about a sci-fi scenario that presupposed a lot of things but not anything that wrote code for you to the extent that society found value in dedicated code librarians. The state of AI today has nothing to do with re-inspecting that future world in light of the last 3 years of GenAI progress.
I'm a SRE and encountered this recently. To prevent DDoS, there is a buffer setting on the kernel that will limit the number of pings (a few settings actually). So if you have a group of machines that all ping a single destination at once, it's very possible to have some that fail to get a reply.
Relatively speaking, it wasn't that bad. It took a few weeks of getting trouble tickets with no root cause, and a bit of googling. But management wasn't okay with fixing the root cause, instead they just increased the timeout/retry window.
Wow. That's a classic. We were quite motivated because we were the ones that got the automated alerts. I still see them in my nightmares: "chopper is down". The machine was called chopper, I'll never forget, it's been close to 30 years. My buddy Jasper and me spent multiple nights trying to track it and when we finally found it we still couldn't believe that that was it. But a simple swap was proof.
if it wasn't an Arnie reference, could it have been a Stand By Me reference, "Chopper, sick balls"? or Eric Bana's Chopper: "if you keep stabbing me, I'm gonna die"?
clearly, i'm the type where everything is a movie reference, or it's a missed opportunity
I've always assumed that in situations like this, a traceroute is better. You can get more information simply by reaching the next stage in the trace, even if you're given zero information beyond "I'm now at the next server".
Nothing says the Time Exceeded packets elicited by your traceroute have to follow the same path back to you that you initiating packets followed out. It's a convenient fiction to think about IP networks acting like circuit-switched networks and in most LAN and small WAN applications they do. Mostly you can get away with thinking that way in more complex networks, too. When you end up in a situation where path asymmetry is causing you grief, thought, it's nice to have the understanding that each datagram can have a unique routing destiny.
I had an experience recently setting up a third-party VPN where the echo responses were being delivered to the correct (host,interface) but with the wrong destination address (not the same as made the request)
I’ve had to explain this over and over throughout my career. The only way to know if something is accessible is to try the exact endpoint and protocol. Even application-aware firewalls will mess with things at times.
Yes, you need to test the exact protocol you want to use. This means tcping/curl, TLS with proper certificates and SNI domains, etc.
However, just as you make sure that the power supply actually supplies power before dismantling something that refuses to work down to the last washer, repairing network problems should start with the basics. Simple test that does not work, or shows something nonsensical, is a great hint that you forgot something, or should start digging elsewhere.
Every time I've had to fight with path MTU discovery not working I've cursed the people who block all ICMP, though. If ICMP echo / echo-reply is the problem just block that. At the very least, allow destination unreachable / fragmentation needed thru (type 3, code 4).
I've worked in gigs that wanted that. They were all about segmentation, but wanted ICMP echo / response available throughout.
Edit: I wonder if any "enterprise" firewalls do ICMP echo proxying. Having the firewall replace the payload would remove some of the tunneling capability (thought I assume you could still finagle a side channel by just timing the packets) but would also eliminate some of the utility (since being able to craft the payload provides a way to test for specific bit patterns in packets causing problems).
It’s been years but I’ve likely used NAT to redirect ICMP pings so the local firewall responds rather than whatever boat they were trying to reach.
Systems change - a server that once used to respond to pings may no longer do so, but client software may not be updated to stop doing pings before connecting to the actual service on the server. In an ideal world the client code would be updated, in practice: hello firewall.
Still working on building a local delivery service in NYC using electric Cargo Bikes, at https://hudsonshipping.co. We're aiming to launch our first pilot in early 2026 in Brooklyn!
Any idea why Spain and Portugal have such small isometric zones? An obvious factor is the mountains in the North, but I'm surprised there isn't easier access between the Iberian peninsula and France
Yes, it is the mountains. Tunnels are very expensive. The French TGV lines only reach the Spanish border at Hendaye on the west coast and Perpignan on the east. The fast line to Barcelona is quite recent.
In northern Spain, there is a slow train line that connects Barcelona with Galicia called the "tren estrella," but it stops everywhere and uses old infrastructure, so it is slow. Traveling to Madrid is always fast with the newer AVE lines, and more are being built.
No idea about Portugal. I guess that it is the same situation, and those routes are covered by buses.
Then you have the Pyrenees between the Iberian peninsula and France. Aside from being mountainous, also fairly low population density, and rather poorer regions to the point that you'll find abandoned villages.
So, the only sensible connections are along the Mediterranean (Catalonia) or the Atlantic coast (Basque Country).
Another reason is that Spain has high-speed rails only since the nineties.
IRC, just in the recent years, there was a high-speed connection built to the Basque Country (from Madrid). From the Basque Country you can go to France with a light rail.
So, the only high-speed connection is along the Mediterranean.
In a way, it's a center/periphery problem.
France is prioritising connections from/to Paris, Spain from/to Madrid.
Unless either side improves the network at their side of the border, it also makes limited sense to improve yours.
Severe lack of investment. While Spain was building one of the largest high-speed train networks in the world, we were closing down existing lines and building enormous highways.
So running a SQL query and returning GeoJSON?
Sorry that didn't work - hopefully you can work on another system like this soon, with someone who actually understands the benefits of tiles!
Op-ed: No. You don't need Geoserver. The main benefit of using Geoserver is operating a service that provides many different kinds of layers to different users, and being able to manage them through a web interface. Every time I have deployed Geoserver I have regretted it.
Geoserver does not really do much directly: it's just a kind of web glue for various kinds of backends. So for tile generation, you're better off using Martin (https://maplibre.org/martin/martin-cp.html) or similar.
Yes but I actually didn’t end up building any of the tile logic in python. I used PostGIS and Postgres. Each query is like 25 lines and supports polys and lines out the box.
I've found some waterways traffic data, but it seems to be previous day usage data, and not real-time usage or position. I'll dig to see if there is some info, as that would add some value to the dashboard for sure!
Ah, I was missing that bit of information, and it explains why I was seeing these two values. Thank you stevage, I'll update the post with this info and cite you.
Thank you - I will watch that with my coffee. I also realize this means I set the extent to 512 in the mvt file - that means I won't benefit from that subpixel precision you mentioned - I'll update the code and test it out
Hi HN!
This is my first blog, inspired by many posts read here.
I attempt to explain how Vector Tiles in GIS tools are constructed from scratch, and how easy it is. This also sets up future blog posts about MapLibre tiles as well as some tricks in the backend to pre-generate the MVT files to serve them up quickly, and update tiles for real time updates on a map.
I'll be happy to answer any questions or feedback!
I'm not entirely clear on why you had to go all the way to "from scratch". Are there really no libraries in Go that can generate tiles more conveniently?
(I've been basically a full time freelance Mapbox consultant for the last 7 years. When I'm generating tiles, it's either statically (tippecanoe), through PostGIS, or occasionally using a JavaScript library. Never had to get my hands quite as dirty as you here...)
It is a real shame that Mapbox GL JS doesn't support GeoJSON vector tiles, which makes a lot of this stuff easier. Yes, they're not as compact as PBF, but for ease of development, debuggability etc, they're a great step along the way and flatten the learning curve. (From memory, OpenLayers does support them...)
I have not looked at PMTiles, so far everything holds nicely generated in memory, so I haven't had the need to store this information. I'll keep it in mind.
Any idea why Mapbox GL JS doesn't support GeoJSON vector tiles?
> I have not looked at PMTiles, so far everything holds nicely generated in memory
For a planet-scale project that I worked on, a multi-layer PMTiles set generated from GeoJSON by Tippecanoe reduced the amount of storage needed by about 60% vs. MVT, at the cost of a longer build time. Result was a single file served by MapLibre on a static web server, no tileserver needed.
The storage savings allowed us to add 3 additional zoom levels on the same cheap, storage-constrained VPS host that ran the web server. We considered moving it to S3, which would be much easier with PMTiles since it's just a single file, but it would've only cost us more money and we didn't need edge caching.
As far as I know, the design of Mapbox GL JS was very heavily geared towards their own needs of producing high performance maps that would be loaded on millions of devices. Obviously they'd never use MVT in that use case, so they didn't bother supporting it.
There are lots of areas where they could have made the lives of developers a lot easier. Another that comes to mind is just how hard it is to look inside vector tiles - they don't really provide much in the way of tools for inspection, etc. I had to build https://stevage.github.io/vector-inspector/ and https://www.npmjs.com/package/tileinfo for instance.
hey, longtime Mapbox employee here. I appreciate all the work you're doing here to help people wrap their heads around vector tiles! This is an old technology at this point, and as you've explained, there are robust tools for moving from GeoJSON to tilesets. It's cool to pull apart the nuts and bolts of a thing (and the Mapbox Vector Tile Spec is open) but there are easier ways to accomplish this objective.
A question for you:
> Obviously they'd never use MVT in that use case, so they didn't bother supporting it.
What does this mean? Mapbox GL (JS and native) both support MVT, of course--that's why we created it! Perhaps you were referring to something else? Higher in this post I see a reference to "GeoJSON vector tiles" and I'm curious what that means. GeoJSON is very convenient (Mapbox helped support its enshrinement as IETF RFC 7496), but one of the hard parts of tiling is slicing features and knowing when to simplify them. Once you've done that, you might as well cram it into a protobuf or other highly efficient container. When you pass Mapbox GL a GeoJSON, it actually cuts the geometry into tiles in memory and uses those for rendering.
Some other general notes:
- the process of tiling is lossy (or should be). if you zoom out to see all of north america, your GeoJSON of neighborhood address points is going to overlap. you should drop most of them from the low-zoomlevel tiles. Tippecanoe does this in brilliant and highly tuneable ways. This applies to geometry simplification, too. Folks should keep this in mind when doing size comparisons.
- map tiling is fundamentally about moving compute into preprocessing and assembling geometry from highly parallelized fetches. MVT is a technology built on and for S3-like services. it's been exciting to see new approaches to this problem that offer lovely ergonomics around deployment etc, but if you have cloud infra, a hot cache, and are optimizing for performance, MVT remains hard to beat
- we continue to research additional optimizations for VT, but the technology has stood the test of time, and proven useful in many different contexts beyond map rendering, including routing and geocoding
Ugh, dumb typo - it was late. I meant "Obviously they'd never use GeoJSON in that use case".
> Higher in this post I see a reference to "GeoJSON vector tiles" and I'm curious what that means.
It's what it sounds like: vector tiles, but instead of protobuf, the data is simply passed directly as GeoJSON. Really convenient for a use case like a server that generates data on demand: easy to generate (ie, it avoids all the difficulty of OP's post), easy to inspect in the browser for debugging. Only downside is it's less efficient space-wise than protobuf. So it's useful as a first step for a proof of concept (to be replaced by MVT), or in a case where the size doesn't matter.
>Once you've done that, you might as well cram it into a protobuf or other highly efficient container.
I'm disputing the "you might as well" bit for many use cases. :) (Again, I think Mapbox is very geared towards large scale uses, but a lot of the internet is small and bespoke).
>MVT is a technology built on and for S3-like services.
It's interesting that you say that. My experience, having been down this road a few times, is that serving MVT from S3 is generally a pain that I don't recommend for new clients. It takes some pretty specific AWS configuration, and the process of uploading thousands of individual files is slow and error-prone. (I wrote a guide on it once but can't find it now).
Yeah it's a good solution for large-scale uses (again...) but not good for the average user.
PMTiles seems like a pretty compelling alternative for those scenarios: ship one file instead of thousands, and rely on HTTP range requests. The downside I ran into is that not all "S3-like services" support that.
In practice, I recommend either hosting data on Mapbox/MapTiler/whoever is cheapest this month if the volumes are low, or setting up a tiny tile server. Even a tiny server is sufficient for serving tiles, and costs a fraction of what Mapbox charges (especially since Mapbox's change to charging per square kilometre, which is absolutely cost prohibitive for sparse data).
>we continue to research additional optimizations for VT,
Can you elaborate? The spec (https://github.com/mapbox/vector-tile-spec) has not had an update in 4 years, and since MVT v2 did not include any substantive changes, the spec is essentially unchanged since 2014. In 2018, a working group for a version 3 was announced (https://github.com/mapbox/vector-tile-spec/issues/103) but then apparently quietly abandoned only a couple of months later.
Didn't mean to imply that tiling is trivial--our initial business model was focused on taking care of that difficulty for our customers, after all, and it wouldn't have made sense if we didn't think we were delivering value.
I will defer to your experience re the utility of tiled-but-still-GeoJSON as a sensible middle ground. I think you're right that we haven't seen this as an area that merits significant attention--it's sort of "not worth optimizing yet [geojson]" or "worth optimizing [MVT]". But I can see how there could be middle grounds in some scenarios.
PMtiles is what I had in mind when I mentioned ergonomics. Brandon's delivered a very compelling approach, as I hope I conveyed to him at CNG earlier this year. The lack of fully specified behavior re range requests is a lingering concern, as you acknowledge, and there are some other areas like incremental updates where having a huge mess of tiles still looks pretty good. But I think it's fair to say that it's overperformed as a solution, and I understand why people are excited about it and the other cloud-native geo formats that have emerged in recent years. Decades ago, Mapbox founders were at DevSeed writing PHP--there will always be some sympathy around here for "just upload it" as a deploy story!
I can't talk about the optimizations we are investigating, but I can at least acknowledge some of what makes the problem so hard (and the update schedule so slow): MVT is quite good, and backward compatibility is a pain, especially when you're optimizing for bundle/binary size (a particularly hard problem when your competitors get to preinstall their libraries on every mobile phone in the world) and working with a customer base that's preintegrated, in every way and to every extent imaginable, with an existing ecosystem. There is a reason people still use shapefiles! Though I hope MVT's reputation remains a bit better than that...
It's weird, I've done an absolute ton of work with random geospatial data from all kinds of sources (see opentrees.org), but when someone asks what format to supply data in, I often suggest Shapefile. There's a kind of rugged simplicity to it, like an old Nokia. It rarely goes wrong in strange ways, everything supports it, and its limitations (especially file size, field name length/casing, geometry mixing etc) tend not to be show-stoppers.
GeoPackage turned into such a complex beast that you have no idea what's going to be inside, I tend to avoid it at all costs.
I understand that you probably wanted to implement this from scratch but did you take a look at tippecanoe? https://github.com/felt/tippecanoe originally by mapbox but got forked by felt.
