Oh heck yeah now this is what I'm talking about! Data link layer shenanigans!!
For that of you who have no idea what the Address Resolution Protocol (ARP) is, to a first approximation it is the missing link between your MAC address (what Ethernet runs on) and your IP address (what Internet runs on). After your IP routing table, which you can see with `route`, decides what immediate next IP address to forward a packet to, then your ARP cache, which you can see with `arp`, will decide which MAC address that IP correspond to (usually your router's local IP address).
Knowing that ARP even exists is considered mild esoterica today, to say nothing of its IPv6 sibling the Network Discovery Protocol. But even though you don't use it much in modern cloud-based deployments, knowing it exists makes your bird's eye view of how the whole network stack works that much more holistic.
Not sure I'd agree about it being esoteric. Understanding or at least knowing about ARP is still very much essential for people in networking. arping is a very useful tool for seeing if machines on the same network segment are up and just not responding to ICMP pings. Anyone looking at tcpdump/Wireshark dumps will run into it sooner or latter.
It is true that software engineers may sooner run into it when debugging their home network than their application though as cloud and traditional networks are very different.
> For that of you who have no idea what the Address Resolution Protocol (ARP) is, to a first approximation it is the missing link between your MAC address (what Ethernet runs on) and your IP address (what Internet runs on)
More commonly, this is 802.11 nowadays, though I guess ARP would be used at any data link layer protocol when the host needs to get an IP: https://en.wikipedia.org/wiki/Data_link_layer
Oh, I see your point now. Yes, the direct parallel of "MAC is to Ethernet as IP is to Internet" is an imprecise one, intentionally so: I remembered being really confused about Ethernet vs. Internet when I learned all this stuff, and this parallel was another good-enough approximation at the time for me to continue learning.
What follows is non-expert commentary and I probably get several details wrong. But here is my deeper understanding:
In reality, Ethernet is a really old protocol (like Metcalfe started working on it back in 1973), and the way we use it today is very different from the way it was originally imagined. In the original paper, you would hook up say 10 computers to a single giant cable and just have Computer A send data to Computer B by sending that data to _everyone_ and letting anyone who wasn't Computer B just quietly drop the network traffic. That's a really elegant design, in the same way a rock is a really elegant weapon, but these days we usually plug any Ethernet devices we have into an Ethernet switch and that does most of the filtering for us -- moving us from a complete graph where everyone is connected directly to everyone back to a spoke-and-hub design with the switch as the hub. But the original Ethernet _did_ end up using MAC addresses as the way to distinguish Computer A from Computer B, and so the analogy stuck in my brain as a "wrong but useful" model.
Now 802.11, aka wifi, comes right out the box with a spoke-and-hub model. All your traffic goes through your wireless router/switch/modem/whatever it's called. Even when you want to share things with something like BitTorrent or Syncthing on the local network, Computer A's wireless network card doesn't usually directly transfer files to Computer B's wireless network card - it goes from A, to router, to B. There's nothing technical I'm aware of that stops this direct A-B transfer, since we know wireless cards can both pull and push data, but you would have to do some networking legwork to get them to be on the same subnet to do so, which probably involves a lot of manual fiddling with IP addresses or something.
But 802.11 also emerged into a landscape where MAC addresses are ubiquitous, so it still uses ARP to translate between IP and MAC for reasons I'm not entirely sure of (probably practically that MAC addresses change a lot less often than IP addresses to).
The main reason for wireless using MACs and ARPs and the like is it makes it more directly interoperable with wired networks. You can just bridge a LAN directly to a WLAN and expect the clients to not realize they are a different type.
If it really is becoming obscure that's a shame, I'm not sure how you could pretend to understand how IP networks work without having at least cursory familiarity with it.
If I was hiring a network technician, I definitely wouldnt hire someone who didn't know what ARP is. It's too easy and fundamental to the field. It took all of like 3 weeks at the trade school I attended to cover IPv4, MAC, ARP, basic routing protocols, TCP, and UDP, and we were definitely chilling. Understanding those things isn't complicated. You just need to know what computers are, and what "networks" are, then it all very easily clicks into place.
I had a job interview today and they asked me to describe TCP and udp, and when I did they told me not everyone knows it nowadays. I guess I can see why, if you're a web dev you never have to worry about such things.
For that of you who have no idea what the Address Resolution Protocol (ARP) is, to a first approximation it is the missing link between your MAC address (what Ethernet runs on) and your IP address (what Internet runs on). After your IP routing table, which you can see with `route`, decides what immediate next IP address to forward a packet to, then your ARP cache, which you can see with `arp`, will decide which MAC address that IP correspond to (usually your router's local IP address).
Knowing that ARP even exists is considered mild esoterica today, to say nothing of its IPv6 sibling the Network Discovery Protocol. But even though you don't use it much in modern cloud-based deployments, knowing it exists makes your bird's eye view of how the whole network stack works that much more holistic.