Sounds like this could potentially be some defective RAM. Memtest86 can boot from UEFI directly, so it should hopefully show up in BDS. A run should tell you what regions of RAM are bad, if any.
Is there any way to improve upon the fit if we know that e.g. y is n times as noisy as x? Or more generally, if we know the (approximate) noise distribution for each free variable?
Yeah, you can generally "whiten" the problem by scaling it in each axis until the variance is the same in each dimension. What you describe is if x and y have a covariance matrix of like
[ σ², 0;
0, (nσ)² ]
but whitening also works in general for any arbitrary covariance matrix too.
> Or more generally, if we know the (approximate) noise distribution for each free variable?
This was a thing 30 odd years ago in radiometric spectrometry surveying.
The X var was time slot, a sequence of (say) one second observation accumulation windows, the Yn vars were 256 (or 512, etc) sections of the observable ground gamma ray spectrum (many low energy counts from the ground, Uranium, Thorium, Potassium, and associated breakdown daughter products; some high energy counts from the infinite cosmic background that made it through the radiation belts and atmosphere to near surface altitudes)
There was a primary NASVD (Noise Adjusted SVD) algorithm (Simple var adjustment based on expected gamma event distributions by energy levels) and a number of tweaks and variations based on how much other knowledge seemed relevant (broad area geology and radon expression by time of day, etc)
It's hard to adopt something that schools don't teach. I know someone who graduated from UCI with a CompSci degree with a specialization in networking, just before the COVID19 pandemic began. He recalled that the networking courses he took did not cover IPv6 at all, except to describe the address format (i.e. 128 bits, written as hexadecimal, colon-separated). Everything he learned about IPv6, he had to learn on his own or on the job. A standard that has been published for over two decades, heavily used for over a decade, and critical in the worldwide growth of the Internet, was treated as an afterthought by one of the premier universities in the US.
Obvious disclaimer: This is a sample size of 1, and an anecdote is not data, yada yada. I'm not involved in academia, and have no insight into the adoption of IPv6 in CompSci networking curricula on a broader level.
Meanwhile, I was taught and practiced IPv6 in 2003-5 in engineering school (France).
As of 2024, IPv6 deployment in France was >97% mobile and >98% residential due to not being required for obtaining a 5G radio license (and then v6 simply carried downward to being available on 4G) + every ISP that provides FTTH also providing v6.
German situation is mostly/rarely/never. Small businesses have their DSL line where their cheapo router will announce an IPv6 prefix which almost all ISPs over here provide. Medium to large businesses usually have some braindead security policies that include switching off all IPv6 functionality in devices.
Ink on paper, where I work. There have been court decisions that have seen Fax as "remote copying". And said that those remote copies only had any legal value if there was an actual paper original. Thus the workflow always has to involve paper that is then archived as paper in a folder...
I once had to print a form and fax to a company with a signature and the instructions said specifically that "signing with a computer and sending digitally is not allowed".
I just signed with macOS Preview, applied some random noise filter and used a one-off online fax service. ¯\_(ツ)_/¯
> Medium to large businesses usually have some braindead security policies
what's the argument behind that? are they scared they might configure their firewall bad and have no NAT to safe them from accidentally making all devices public?
It comes from the same place as "passwords expire every 30 days".
People don't understand something and just apply the most annoying rule possible.
The craziest one I saw in Germany was "cookies are allowed, localStorage is not", that was for our app. CTO overrode the CISO on the spot and called him an idiot for making rules he doesn't understand. Interesting day.
Usually there is no official justification given, just a list (in excel...) of security requirements that have to be ticked off. One of them is "Disable IPv6".
I've heard some ex-post justifications, make of them what you will: Existing infrastructure like firewalls, VPNs and routers might not be able to handle IPv6 properly. Address distribution in IPv6 is unpredictable. No inhouse knowledge of IPv6. Everything has an address in IPv6, so the whole internet can access it. No NAT in IPv6, so it is insecure. IPv6 makes things slow.
Tbh it’s is a huge PITA with little practical benefit. IPv6 is the Perl 6 of networking.
Many of the big benefits are things that don’t deliver anything that folks are lacking. You also need to understand how you fit in the overall universe more.
An example for a small environment: I've got the whole homelab on unique ipv6 range. Whatever VPN connection happens to another network, I'll never have range collisions or need any fancy rewriting. Also the DNS will point at a specific address on my network, never at a random 192.168.x.x in a network I happen to be connected to.
You’re not wrong, but I have been running complicated multi-site VPNs with a small homelab multi-subnet / VLAN setup for 25 years and still have yet to have a collision.
My home network is dual-stack these days, but because my IPv6 prefix is dynamically delegated by my ISP, I actually use site-private IPv6 addresses for all my internal servers and infrastructure.
The thing is though, I don’t even need IPv6. Comcast Business broke my delegation for six+ months and I literally didn’t even notice.
IPv6 tried to do way too much. The second system syndrome was strong. It’s no wonder folks are annoyed at the complexity, and as long as IPv4 continues to works for them, they aren’t particularly pressed to adopt it.
> You’re not wrong, but I have been running complicated multi-site VPNs with a small homelab multi-subnet / VLAN setup for 25 years and still have yet to have a collision.
And I've been in corporate IT networks with mergers/acquisitions where both organizations involved had 10.0.0.0/24. Ever have NAT inside a company? Fun stuff. (Thrown in some internal-only split-horizon DNS too.)
Then there's the fact that in the COVID period we had IPs for VPN clients (172.*) in the same range as what some developers used for their Docker stuff. Hilarity.
Even supposedly prosumer gear sucks at ipv6. The ubiquiti situation was awful about a year ago. I got a dynamic prefix and wanted to setup ULA. Maybe I was dumb, but I couldn't find any way to do it.
Heck, I couldnt even see which prefix I was handled, nor could I see any ipv6 address anywhere in the gui. This was with a self hosted up to date controller though. YMMV.
Part of it being that a lot of ISP's don't have static prefixes, they do get rotated pretty often and have no guarantee of CIDR size that you're going to get. By default my ISP will only give a single /64. You have to go out of your way to request more subnets and there's no guarantee that the ISP will honor that request.
It's really problematic to try and base a non trivial network setup, when you have no guarantee of how many subnets you can run. Today I've got 256. Tomorrow it might be 16. Or 2. Maybe just 1 again. ISP's can be weird when they smell monetization dollars in the water.
So I have to run a ULA in parallel to the publicly accessible networks specifically for internal routing, and then use a DNS server to try and correct it. Which works great! ...except when you run into this little niche operating system called Android. Which by default doesn't obey a network provided DNS server if you've got privacy DNS enabled. So if I've got guests over and I want them on a network in my place to access some sort of internal resource, then I've got to walk them through disabling privacy DNS.
Either that or I need to go out and buy a domain... for my internal network...and then get a TLS certification for my private internal domain.
I get how IPv6 can be great. But a lot of the advantages are also overhead I don't want to deal with.
Short hand is a good example; I've lost count at the number of times I've typo'd short hand addresses because my eyes skip over a colon. At this point I've gotten into the habit of just writing out the whole address, leading 0's included because the time saved from not making a mistake reading the address often faster overall then making mistakes with shorthand.
> So I have to run a ULA in parallel to the publicly accessible networks specifically for internal routing, and then use a DNS server to try and correct it. Which works great! ...except when you run into this little niche operating system called Android. Which by default doesn't obey a network provided DNS server if you've got privacy DNS enabled. So if I've got guests over and I want them on a network in my place to access some sort of internal resource, then I've got to walk them through disabling privacy DNS.
This also sounds like it would be a problem for v4? I'm not clear on how this is a v6 problem. If I'm picturing it correctly, it's a difference of handing the guests a local v4 address vs disabling privacy DNS and handing them a DNS name. I'd think the latter would be easier
Using a public domain for TLS certs for private networking is pretty standard in /r/selfhosted and /r/homelab at least.
Fair point on ISPs handing out /64 prefixes, but this is the first I've heard of them varying the prefix length once you know what you've got. I don't doubt it though
> Either that or I need to go out and buy a domain... for my internal network...and then get a TLS certification for my private internal domain.
TBF, if you are on HN that should be extremely simple for you. I use a subdomain of my primary email domain I own, and use LetsEncrypt to issue TLS certs on my internal network. Well beyond the means of my mom and sister, but probably pretty easy for most people here.
It still helps. I have a 1U in a colo which gives me a /64 for ipv6 and ~5 addresses for ipv4. I just set up a dual stack kubernetes cluster on 6 virtual machines. When I want to ssh into one of the machines, my options are either:
1. Use IPv6 which works and goes directly to the virtual machine because each virtual machine grabs its own address from one of my 18446744073709551616 addresses.
2. Use IPv4 and either have to do a jumphost or do port forwarding, giving each virtual machine its own port which forwards to port 22 on the virtual machine.
3. Use a VPN.
I have all 3 working, but #1 was significantly less setup and works the best.
Also being able to generate unique ULA subnets is super nice.
Really using port 22 is very ill advised anyway because you will get constant nuisance brute force attacks (accomplishing nothing because you're using keys or certificates I hope) but still eating up cycles for the crypto handshake.
By that same logic, using IPv4 is ill-advised because I could easily give the ssh endpoints their own IPv6 addresses, avoiding the need to hide behind non-standard ports. Scanning through 18446744073709551616 addresses is going to be a lot slower than scanning through 65536 ports.
You can't just list out all the DNS names. The three ways that names get discovered are:
1. You listen on IPv4 and someone probes all the IPv4 space and your server announces "Hi, I am web123.example.com" or similar in its responsible
2. You have HTTPS on the server and the HTTPS address ends up in the certificate transparency logs.
3. You have a public service on that server and announce the address somewhere.
But when you have billions of IP addresses, why does SSH need to listen on the same address as HTTPS or anything you're running publicly? It's also infeasible to probe the entirety of IPv6 space the way you can probe all of IPv4, even though we're only assigning addresses in 3/65535 of it right now.
I've had SSH open on a static v6 that isn't even SLAAC or temporary, it's not my/58::1 but not far off and in DNS, and I have not in 8 years seen a single scan or connection attempt over IPv6 (other than myself). This is not to say there is no risk, but it really is a night and day difference.
Really? I get somewhere in the region of none to barely any, depending on the server.
I mean, yes, you'll get a constant stream of them on IPv4, but why would you run a server on v4 unless you absolutely needed to? The address space is so small you can scan every IP in 5 minutes per port, and if you have my v4 address you can enumerate every single server I'm running just by scanning 65k ports.
Meanwhile, on v6, even the latter of those takes a thousand years. How would people even find the server?
If you are an ISP running dual stack ipv4 with NAT plus ipv6, the more connections happen via ipv6 and the more traffic happens via ipv6, the better, because it doesn't have to go through the NAT infrastructure which is more expensive, and cost scales with traffic (each packet needs its header to be modified) and number of parallel open connections (each public v4 address gives you only 65k port numbers, plus this mapping needs to be stored in RAM and databases).
7621 devices include hardware NAT. And anything Qualcomm in the recent past does. Most home WiFi 5 and above routers can do hardware NAT just fine. Hardware NAT allows for using cheap and old cpus for CPE. ISP hardware is a different story. Some decent routers that can do that which don’t cost a lot.
> Not really, this is only true for mobile devices.
Tell that to my fixed line provider, with their CGNAT ... And its just about every provider in Germany pulling that crap. O, and dynamic IPv6 pre-fix also, because can't have you run any servers!
Yes, plenty of ways to bypass it but when you have ISP's still stuck in 1990's attitude, with dynamic IPv4/IPv6, limited upload (1/3 to 1/5 of your download), etc ...
> Adopting IPv6 doesn't alleviate the pain of IPv4 exhaustion if you still need to support dual-stack.
Sure it does: the more server-side stuff has IPv6 the fewer IPv4 addresses you need.
If you have money (or were around early in the IPv4 land grab) you have plenty of IPv4 addresses so can give each customer one to for NATing. But if you don't have money to spend (many community-based ISPs) you have to start sharing addresses (16:1 to 64:1 is common in MAP-T deployments). You also have to spend CapEx on CG-NAT hardware to handle traffic loads.
Some of the highest bandwidth loads on the Internet are for video, and Youtube/Google, Netflix, and MetaBook all support IPv6: that's a lot of load that can skip the CG-NAT if the client is given a IPv6 address.
If you can go from 1:1 to 16:1 (or higher) because so few things use IPv4 that means every ISPs can reduce their legacy addressing needs.
> The widespread deployment of NAT and VPNs has counter acted the market forces that were assumed to make IPv6 appealing.
Tell that to everyone who is behind CG-NAT and has issues with (e.g.) video games. Or all the (small(er)) ISPs that have to layout CapEx for translation boxes.
Which has led to every game needing a central server running, forcing centralization where p2p used to work great. Also how Skype was able to scale on a budget, something now blocked, forcing you to raise money for more ideas than before. Running a matrix(?) node should be as simple as clicking install and it's just there, next time you're with your friends, nfc tap or whatever and your servers talk to each other directly forever going forward. But nope, there always is a gatekeeper now and they need money and that poisons everything.
I don’t think VOIP was a major factor in game centralization. The big one was selling cosmetics (easily unlock able server-side in community servers), and to some extent being able to police voice chat more. Major game publishers didn’t want to be in the news about the game with the most slurs or child grooming or what not.
Central servers are useful for more than just NAT hole-punching. They’re also great as a centralized database of records and statistics as well as a host for anti-cheating services and community standards enforcement.
Peer to Peer games with no central authority would be so rife with cheating that you’d only ever want to play with friends, not strangers. That sucks!
> Peer to Peer games with no central authority would be so rife with cheating that you’d only ever want to play with friends, not strangers. That sucks!
Back in the the day RtCW had a server anyone could run and you could give out the address:
If you can run your own server then that's still a central server. That still lets a community of people work with a central authority. It's just a different authority from the game's publisher.
In that sense, Mastodon is a centralized service because it's on someone's computer. That's not really what people mean by central. They mean we're increasingly reliant on game companies for networking infrastructure.
Is that all IPV4s fault? I don't think so. But it complicates things
I think you're muddling things up more than they need to be. A peer-to-peer game is one in which players connect directly to each other but neither is the host and there is no dedicated server. Game state is maintained separately on each player's computer and kept in sync by the netcode. Since there is no single source of truth for the game-state, so players are free to cheat by modifying the game's code to lie on their behalf. There is also the side issue of bugs in the game code causing the game-states to become irreparably desynchronized.
All of these issues are solved by having a central server for both players to connect to. Whether that server is owned by the game's publisher or by an open-source community is irrelevant from a technology standpoint. However, the prevalence of IPv4 networks and stateful NAT firewalls is relevant because it privileges those central servers over true peer-to-peer connections.
I don't disagree with you, I just read your comment as deriding people who think hosting their own game servers is meaningful, because it's similar to a company server. Sounds like you didn't mean it that way.
Don't put words into my mouth! I never said I didn't care about peer to peer networking and peer to peer gaming. I said it sucks if your only option to avoid cheating is to play with friends.
If you only care about gaming with friends, then peer to peer is an excellent way to do that (assuming the game doesn't have any synchronization issues, which some peer to peer games do).
So we acknowledge v4 and CG-NAT are a problem but don't want to use the already available solution because game developers took it upon themselves to DEFEAT NAT :)
That just reminded me of a peer protocol I worked on a long time ago that used other hosts to try to figure out which hosts were getting translated. Kind of like a reverse TOR. If that was detected, the better peering hosts would send them each other's local and public addresses so they could start sending UDP packets to each other, because the NAT devices wouldn't expect the TCP handshake first and so while the first few rounds didn't make it through, it caused the NAT device(s) to create the table entries for itself.
Was it Hamachi that was the old IPX-over-IP tunneling? I'm fairly sure it used similar tricks. IPX-over-IP is also done on DOSBOX, which incidentally made it possible to play Master of Orion 2 with friends in other continents.
> That just reminded me of a peer protocol I worked on a long time ago that used other hosts to try to figure out which hosts were getting translated. Kind of like a reverse TOR. If that was detected, the better peering hosts would send them each other's local and public addresses so they could start sending UDP packets to each other,
If that's the VOIP thing, yes, lots of people came to similar methods. That particular thing was for exchanging state, not VOIP or tunneling, so as long as participant groups overlapped it didn't really need a fixed server to be the middle which was handy for our purposes, although long network interruptions could make reconvergence take a while.
Does make me chuckle that so many people had to be working around NAT for so long and then people are like "NAT is way better than the thing that makes us not have to deal with the problem at all." Just had a bit of NAT PTSD remembering an unrelated, but livid argument between some network teams about how a tool defeating their NAT policies was malware. They had overlapping 10.x.y.z blocks, because of course they did :)
Nat hole punching works... most of the time. There are many edge cases and weird/broken networks which you just can't work around in standard ways. You get to see all kinds of broken setups if you work at VoIP providers. That's why everyone will use a central proxy server as the last resource - you'll mostly notice it only because of a higher ping.
IPv6 is used on mobile networks since there aren't enough IPv4 addresses. Some of these mobile networks are so big there aren't even enough private IPv4 addresses for their CG-NAT private side to fit, leaving the only clean solution being NAT64/DNS64.
Why would CGNAT be deployed as a response to IPv6 on mobile? I don't understand the logic there. CGNAT is deployed due to a shortage of publicly routable IPv4 addresses. IPv6 was introduced due to having much larger publicly routable space.
No, CGNAT (Carrier-Grade NAT - https://en.wikipedia.org/wiki/Carrier-grade_NAT) is an IPv4 only thing. https://www.rfc-editor.org/rfc/rfc6598 specifies they should use 100.64.0.0/10 for it, to avoid conflicting with the pre-existing private-use ranges. IPv6 removes the need for using CGNAT, as each home router is allocated a public IP (rather than a CGNAT IP) on its public link.
No, CGNAT has absolutely nothing to do with IPv6. CGNAT is nothing more than ISPs not providing a public IP to the gateway on your LAN (i.e. your router). To avoid conflicts with existing ranges, a new ranges for that purpose was allocated. There are different technologies to enable IPv4<->IPv6, none of which care about the existence of CGNAT.
No, NAT64 was invented so v6-only hosts could access v4-only resources. CGNAT was invented so v4 hosts can have a v4 address without having to purchase limited public address space.
It's my dream that one day I'll be able to run an AWS VPC that only has IPv6 for the private subnets and then I'll never have to worry about managing the address space or how many IP addresses each ALB consumes.
Yep, iot would be a tremendously worse security problem if everyone wasn't actually operating a household subnet without knowing it.
When your washing machine, fridge, etc all come with ipv6 5g modems is when your house becomes part of the future IT battlescape between lots of different entities that do not wish you well.
If your dishwasher has a 5G antenna + modem built-in and connects to the manufacturer’s own wireless account then your router doesn’t enter the picture. The dishwasher can happily serve you ads and conduct routine surveillance all day long and the only thing you can do is cut power to the device (until they start including a battery backup for that stuff).
True, but the dishwasher should have its own firewall regardless, and assuming it'll be on IPv4 behind a firewalled NAT is by itself an implementation error.
My point is that you don't control what network the dishwasher is on, the manufacturer does. The dishwasher connects to its own cellular network so that you cannot block any of its ads or prevent it from spying on you.
I’m assuming you don’t know how iPv6 works. With SLAAC every device usually rotates the v6 address every few hours and maintains multiple of these. Each subnet for each customer is huge. With rotating MAC it’s virtually impossible to maintain a connection with an IPv6 only device by just IP address. It’s one of the features of IPv6 that such attacks are not going to be feasible.
I am truely a beginner. I am also annoyed by rotating identifiers for devices on the network since it increases the overhead to differentiate for the purpose of firewall rules. Maybe v6 has an identifier better than MAC that can be handled expeditiously for DNS and IP controls?
Why? My router won’t even let me DMZ a single ipv6 device or open all ports to a single ipv6 device. It will only let me open one port at a time.
different routers have different options, but all of them have come with a pretty strong firewall out of the box, turned on by default, for the last 10 years.
> Enough addresses for what? Nobody needs or even wants all of their devices to have globally routable addresses.
They do if they have applications, such as Xbox/PS gaming applications, broken VoIP in gaming lobbies, failure of SIP client to punch through etc. And if an ISP does not have, or cannot afford, to get enough IPv4 to hand each of their customers at least one to assign to the CPE's WAN port, you're now talking about CG-NAT, which a whole other level of breakage.
We’re supposedly mere years away from superintelligence, but it’s still literally impossible to just send a file between two clients without configuring intermediate network hardware or performing some hack to get around NAT (which can still fail and then require an intermediate server) if both clients are behind CGNAT.
It’s genuinely disheartening to see so many people here not even begin to try to understand how much we’re missing by not having effortless end-to-end connectivity, in favor of expensive cloud services. This literally used to be what the “Internet” is - we’re definitionally not on one without this.
I think the western world very much revolves around:
* The internet
* Linux servers
* Automation
I get your point, but it falls on deaf ears to me since most people don’t feel the benefits until some passionate nerd makes something that scratches an itch.
For a practical example: peer-to-peer sharing like Airdrop is much easier to implement in a world with ipv6.
The world at large doesn't care what I love, correct. But my users care about whether they have to remember that they're supposed to use port bla instead of the standard port foo, which is a common scenario with v4. Not enough addresses, and / or you can't get them to the VM or container or VPN client or whatever that needs them.
IPv6 can often fix these kinds of issues.
Does the world at large care? No.
Do I care? Yes.
Do my users care? Yes, albeit indirectly.
Does my organization care? Yes, in the sense that it removes friction from what it needs the employees to do.
And that's all the justification that's needed, I'd say.
The world very clearly doesn't need to revolve around what I love for IPv6 to be a good thing.
No One believes us on hacker News. It feels very gaslighty. I have never talked to an IT engineer in person that thought IP version 6 in the data center or in the corporate network was a good idea.
I recently passed the CCNA again and they really spend a lot more time on IPv6 compared to 15 years ago. It inspired me to go all in this time and configured my home network with a PD allocation from my ISP. I also came up with some fun labs and even got a IPv6 sage T-shirt from Hurricane Electric.
Any recommended courses? I'm a SWE and never felt compelled for the CCNA but my intersection with networking-related problems seems to continuously increase and I would like to up my game before getting in over my head at work.
I just bought the official exam guide found Neil Anderson’s videos helpful. One thing that bugged me a bit was they spent a little too much time on their WiFi, including the obsolete Airie OS.
This doesn’t hold up. Schools can’t teach everything, especially in a field where innovation happens in the workplace, not the classroom. Should I have learned about LLMs when I was an undergraduate 20 years ago?
This is just further proof that university educations are still not job training. The sooner we disabuse ourselves of that perception the better off society will be.
Higher education is about creating a breadth of knowledge, not specific marketable skills. CompSci is a research field, not job training.
If your friend wanted to learn specific job skills a technical college would be the appropriate setting.
I realize this misperception is perpetuated by the job market but I’m still not surprised at the education provided by UCI and don’t fault them for providing it.
They taught us, they also taught ipv4 in the old "separate address per host" way instead of jumping to NAT, but I think ipv6 is inherently more complicated than ipv4 for the average use case. It's not just a thinking shift.
Separate from that, deliberate decisions were made to make it a "clean slate" without consideration for existing ipv4 hosts. Guess they were hoping the separate stacks would go away eventually, but in hindsight, no way.
> ... but I think ipv6 is inherently more complicated than ipv4 for the average use case. It's not just a thinking shift.
IPv6 isn't all that complicated for most common use cases. Its fundamental concepts and rules are simple. It also obviates the necessity of the complicated workaround called NAT, without which IPv4 is impractical these days.
It's more like the imperial vs metric system debate. If the world hadn't seen IPv4, I believe that we'd all be using IPv6 without any complaints. The real problem is that IPv6 isn't taught well.
> Separate from that, deliberate decisions were made to make it a "clean slate" without consideration for existing ipv4 hosts. Guess they were hoping the separate stacks would go away eventually, but in hindsight, no way.
I'm not sure what to make of this. The presence of the IPv4 stack isn't what blocks the adoption of IPv6 - at least not technically. They can coexist on the same host and function concurrently without interfering with each other. It was designed to operate like that. The actual blocker is the attitude that people hold towards IPv6 - "We have IPv4 that works already. Why should we care about an alternative?". You can see that expressed on this discussion thread itself.
There is one crucial detail that the IPv6 detractors neglect - the scarcity of IPv4 addresses means that IPv4 address blocks are now heavily coveted and therefore subject to moneyed interests. That isn't very good for the health of the open internet, digital rights and equity. They're thinking about individual trees and losing sight of the whole damn forest. IPv6 isn't a solution looking for a problem. It's the solution for a problem that people simply ignore.
The IPv6 spec was being modified up through 2017. It has more kinds of addresses that behave in fancier ways, with one host having multiple. The very first thing you see with ipv6 is your nice memorable ipv4 addr replaced with a long hex string with some ::s thrown in. Local DNS is commonly recommended with ipv6 for that reason, which maybe is just some misguided advice because it sounds crazy. I guess you could assign and memorize ULAs?
NAT is technically complicated if you're looking inside it, but most people aren't, and for them it's really easier to think about. You've got a public and a private, and there's a very strong default that private isn't exposed. People screw up firewall rules all the time or routers have bad defaults, but it takes more deliberate action to publicly expose a port over NAT. Plus you don't need privacy addresses that way (introduced to ipv6 in 2007). I know "NAT isn't security" but for most people, it is.
Still not even sure what the accepted default firewall behavior is in ipv6, cause some people say "ipv6 lets any device do p2p by its own choice" and then when you ask about security, "your router firewall should always default-deny anyway," so which one is it?
> The presence of the IPv4 stack isn't what blocks the adoption of IPv6
It is. Like they say, most technical problems are really people problems, especially this one.
> Local DNS is commonly recommended with ipv6 for that reason, which maybe is just some misguided advice because it sounds crazy.
Many (most?) SOHO routers already run a combined DHCP and DNS server called 'dnsmasq', which supports DHCPv6. IIRC, dnsmasq automatically adds DNS records for hosts to which it gives out a lease. Android computers don't use DHCPv6, so this won't help you access them by name, but how often do you care to directly access an Android computer?
> I wasn't under the impression that SOHO routers normally have DHCPv6 enabled by default.
The fellow I replied to indicated that running a local DNS server on one's LAN "sounds crazy".
My commentary was intended to indicate that it's very common in SOHO networks to already be running a DNS server that automatically adds hostname->address mappings of DHCP clients on that network. It also mentioned that DHCPv6 support is supported by the combined DHCP+DNS daemon used by many (most?) SOHO routers.
My commentary was not intended to indicate that DHCPv6 support is on by default on many or most SOHO routers, only that it's likely to be supported, and that -if supported- it is very, very likely to put hostname->AAAA mappings of DHCPv6 clients into its DNS server, just as it adds hostname->A mappings for DHCPv4 clients.
Ah, I understand. Well having dnsmasq fully automatically run DNS isn't crazy at all if you're using DHCPv6. If you're not, it sounds unreasonable for you to need to spin up your own DNS server.
> If you're not, it sounds unreasonable for you to need to spin up your own DNS server.
If you're using a SOHO router, you're very likely to already be using dnsmasq; a DNS server. In that configuration, if you're using DHCP then you get your hostnames in DNS for free.
If you're not using DHCP and don't have a DNS server running on your network that you have figured out how to update with host IP addresses, then it's on you to select memorable static addresses. [0] This is a long-standing baseline fact of IP addressing for LANs and other private networks.
[0] Nothing prevents you from assigning addresses to your LAN machines in the fd00::/64 prefix starting from 1 (that is, fd00::1) and going up. The fd00::/8 space is for uncoordinated network-local addressing.
I can’t think of any technology where mass adoption was driven by knowledge forcibly inserted into students’ brains by schools… if anything, adoption comes when people realize their out-of-touch curriculum is no longer relevant.
To be clear, degree programs have value, but it’s not in future-proofing students against needing to learn things after they leave school. Ideally it should prepare them and encourage them to do so.
>> I know someone who graduated from UCI with a CompSci degree with a specialization in networking, just before the COVID19 pandemic began. He recalled that the networking courses he took did not cover IPv6 at all...
I am not doubting you, but I feel this story is too hard to believe without adding further nuances...
The issue is that it’s not taught with IPv6 first. Networking courses do all kinds of stuff using IPv4 to demonstrate various protocols on top (e.g. http, tcp, icmp, etc).
Then there is usually a chapter on IPv6 that just briefly covers the differences.
I.e. the exercises all tend to use IPv4 as the foundation so people don’t practice v6
But TCP or HTTP don’t care about the underlying transport. They’re higher level protocols that are payloads to either IPv4 or IPv6. It’s irrelevant what the transport is when dissecting HTTP and very little time should be spent on it.
IPv4 is, for all intents and purposes, still the default transport. It’s also simpler than IPv6 in some regards. When teaching layer 3, it makes sense to teach both, and teach IPv4 first. Though I fully agree that they should be taught with equal emphasis. I don’t doubt there’s a good number of programs out there that don’t into sufficient detail on IPv6.
No, this is wrong and it’s why academia is failing.
IP addresses show up everywhere when you are working with both TCP and HTTP. Sockaddr is all over sockets programming, IPs show up in HTTP headers, etc.
They absolutely care about the underlying protocol because the underlying protocol is how you address the other end.
I've been of the opinion this is one of those "the art advances one funeral at a time." A lot of people are married to IPv4 and its arcane warts and really, really do not want to deal with IPv6 even though most of the core concepts are almost exactly the same thing, except better. I can't imagine anyone who dealt with V4 multicast ever wanting to go back, and I bet they've memory-holed parts of V4 that simply can't be used anymore and so have been turned off for decades(RIP to RIP). Has anyone seen the automated address assignment in V4 ever work? The usual hint it even exists is that if you see one of those addresses it means something is messed up in your Windows host or the DHCP server died.
People complain about dual stacks and all that but with a modicum of planning it is minimal extra effort. Anything made in the last decade has V4/V6 support and unless you're messing with low level network code, it's often difficult to even know which way you're being routed. Network devices pretty much all support using groups of names or addresses and not hard coded dotted-quad config statements now, and have for a while. And that was good practice on V4 networks too.
Part of it is probably that remembering various V4 magic is easy enough to do but feels complicated enough to be an accomplishment. In V6, there is no point in doing most of that because the protocol has so much more automation of addressing schemes. But if you like those addressing schemes, V6 can do them even better. You can do all sorts of crazy address translation on either the network or host id portion, like giving an internal network a ULA that is magically translated to a public network prefix without any stateful tracking unless that is desirable.
I feel there is some analog to DNS in that regard, people who have gotten used to DNS don't give a damn about host IP addresses but some people seem to really like the idea of a fixed address statement. People also seem to be stuck on the idea that NAT creates some kind of security when that's really the stateful tracking that is required for many-to-few translations (thus making firewalls a common place to implement it), not the translation itself. Similar to certificates/pki versus shared keys, yes, one is more upfront effort but that's because it's solving the problem of the Sisyphean task that is the other.
edit: This all reminded me that we lived with dual stacks before, in the IP and IPX days, or DECnet, and that GE Ether-whatever, and those had less in common. IPX mostly died with Netware but it had a number of advantages that wouldn't be bolted on top of IP for years, some of which are present in IPv6. I rather liked IPX and had history gone differently that it used 48-bit addressing would be causing us to discuss whether or not EUID was a mistake or not :)
Exactly. IPv6 was developed in the ivory towers where it was still assumed that everyone wanted to be a full participant of the internet.
But the social/political environment was that everyone just wants to be a passive consumer, paying monthly fees to centralized hosts to spoon-feed them content through an algorithm. For that, everyone being stuck behind IPv4 CG-NAT and not being able to host anything without being gatekept by the cloud providers is actually a feature and not a bug.
We've seen only the world where everything has been adopted to IPv4. p2p technologies strive even under it, but they could really shine with the ability to connect directly between devices. Imagine BitTorrent on steroids, where you don't have peers with assigned IPv4 and seedboxes and everybody else. Torrents are generally faster than usual channels to download things, but with ipv6 it would be far faster than now.
Cloudless cameras streaming to your phone without Chinese vendor clouds, e2e encrypted emails running on your phone without snooping by marketing people and three-leter agencies, content distribution network without vendor lock-ins. The possibilities are impressive if we have a way to do it without TURN servers that cost money and create a technical and legal bottlenecks.
We can't say nobody wants that world because we've never tried it in the first place. I definitely would like to see that.
Don't you think everyone should have the option to be a full participant? Being locked behind cloud providers and multiple layers of NAT with IPv4 means that can never happen, even if consumers want it to.
I was lucky enough to experience the 90's internet where static IP addresses were common. I had a /24 (legacy "class C" block) routed to my home, and still do.
> Exactly. IPv6 was developed in the ivory towers where it was still assumed that everyone wanted to be a full participant of the internet.
IPv6 was developed in the open on mailing lists that anyone could subscribe to:
The criteria presented here were culled from several sources,
including "IP Version 7" [1], "IESG Deliberations on Routing and
Addressing" [2], "Towards the Future Internet Architecture" [3], the
IPng Requirements BOF held at the Washington D.C. IETF Meeting in
December of 1992, the IPng Working Group meeting at the Seattle IETF
meeting in March 1994, the discussions held on the Big-Internet
mailing list (big-internet-at-munnari.oz.au, send requests to join to
big-internet-request-at-munnari.oz.au), discussions with the IPng Area
Directors and Directorate, and the mailing lists devoted to the
individual IPng efforts.
Just like all current IETF discussions are in the open and free for all to participate. If you don't like the direction things are going in participate: as Gandhi did (not) say, “Be the change you want to see in the world.”
One of the co-authors on that RFC worked at BBN: you know, the folks that actually built the first the routers (IMPs) that created the ARPA/Internet in the first place. I would hazard to guess they have know something about network operations.
> But the social/political environment was that everyone just wants to be a passive consumer, paying monthly fees to centralized hosts to spoon-feed them content through an algorithm.
Disagree, especially with the hoops that users and developers have to jump through to deal with (CG-)NAT:
> [Residential customers] don't care about engineering, but they sure do create support tickets about broken P2P applications, such as Xbox/PS gaming applications, broken VoIP in gaming lobbies, failure of SIP client to punch through etc. All these problems don't exist on native routed (and static) IPv6.
Well, with such a description of the 'vices' of IPv6 vs the 'virtues' of IPv4 count me as one who considers himself in full support of the ivory towered greybeards who decided the 'net was meant to be more than a C&C network for sheeple. Once I got a /56 delegated by my IAP - which coincided with me digging down the last 60 metres of fibre conduit after which our farm finally got a real network connection instead of the wires-on-poles best-effort ADSL connection we had before that - I implemented IPv6 in nearly all - but not all - services. Not all of them, no, because IPv6 can make life harder than it needs to be. Internally some services still run IPv4 only and will probably remain doing so but everything which is meant to be reachable from outside can be reached through both IPv4 as well as IPv6. I recently started adding SIP services which might be the first instance of something which I'll end up going IPv6-only due to the problems caused by NATting the SIP control channels as well as the RTP media channels, something reminiscent of how FTP could make life difficult for those on the other side of firewalls and NAT routers. With IPv6 I do not need NAT so as long as the SIP clients support it I should be OK. Now that last bit, client support... yes, that might be a problem sometimes.
So like plugging two laptops together? Honestly curious, I can't recall ever seeing anyone using it and the situations that it seems like it should be good for, like initial configuration of stuff coming out of the box, instead come with instructions for setting specific IPv4 addressing or use DHCP. Possibly a lack of some LLNMR equivalent at the time.
link-local is mandatory for ipv6 to work. Technically everybody you have ever seen is using it. It is unlikely that you know somebody without a cellphone. And as far as I know, all cellphone networks are ipv6 first.
80% of my career knowledge as a devops engineer, systems administrator, and IT engineer has been on the job training. That's just how it works.
The real reason is IT people hate ipv6. They want NAT. They don't want all the security holes and extra complexity. I don't want having to work with a network stack that is poorly supported by some switches and routers.
IPv6 was superceded by NAT a long time ago. It will die a slw and quiet death which is why it is now being ignored by training facilities and experts worldwide.
Oh no, somebody should warn all the ISPs deploying IPv6-native connections with v4 reachable over some fallback technology (464XLAT, DS-Lite, NAT64 etc.) to their hundreds of millions if not billions of customers!
The only ISPs issuing IPv6 only connections are mobile device operators and Telcos. THey are a small subset of ISPs in the world and IPv6 only connections will never gain any traction outside of that world.
I agree it will not die so I retract that statement, but it will never fully replace IPv4 in standard wired internet connections.
Digital Ocean didn’t even have an ipv6 address on by default in the droplet I created last week. It’s just a switch to flip, but I’ll bet the support costs of hobbyists/enthusiasts not realizing they needed to also write firewall rules, make sure ports weren’t open for databases and things like that for ipv6.
My memory of IPv6 is getting waves of support tickets from people who took their (already questionable) practice of blocking ICMP on IPv4, blocked ICMPv6, and then got confused when IPv6 stopped working.
It's a "just doesn't work" experience every time that I try it and I don't experience any value from it, it's not like there isn't anything I can connect to on IPv6 that I can't connect to on IPv4.
My ISP has finally mastered providing me with reliable albeit slow DSL. Fiber would change my life, there just isn't any point in asking for IPv6.
Also note those bloated packets are death for many modern applications like VoIP.
Exactly. Spectrum delivers good IPv6 service in my area. I tried it when I upgraded my gateway. All of my devices are assigned 4 IPv6 IPs, hostnames are replaced by auto assigned stuff from the ISP, and lots of random things don’t work.
I went from being pumped to learn more to realizing I’m going to invest a lot of time and I could not identify and tangible benefit.
The biggest tangible benefit is you don't need to worry about NAT port mapping any more. Every device can have a public address, and you can have multiple servers exposing services on the same port without a conflict.
(The flip side is having a network-level firewall is more important than ever.)
You also don't have to worry about running a DHCP server anymore, at least on small networks. The simplicity of SLAAC is a breath of fresh air, and removes DHCP as a single point of failure for a network.
So the benefit is that you dont need to worry about NAT for a couple of port forwarded services you may use (which might well even use UPnP for auto setup), but the tradeoff is you now need to think about full individual firewall protection for every device on your network?
I'll take full security by default and forward a couple of ports thankyou!
You can maybe connect to everyone over IPv4, but chances are that that path is strictly worse (in terms of latency, P2P reachability, congestion et.c) than a v6 one would be.
For example, two IPv6 peers can often trivially reach each other even behind firewalls (using UDP hole punching). For NAT, having too restrictive a NAT gateway on either side can easily prevent reachability.
I have tailscale on all my mobile/portable devices I use away from home. It punches holes so I don't have to, even makes DNS work for my tailnet in a way I've never been able to get to work the way I want the normal way.
Yes, Tailscale is great, and it does manage to traverse pretty much every firewall or NAT in my experience as well. Quite often, it even does so using IPv6 :)
> those bloated packets are death for many modern applications like VoIP.
Huh? The packet sizes aren’t that much different and VOIP is hardly a taxing application at this point anyway. VOIP needs barely over dial-up level bandwidth.
It's not the bandwidth it's the latency. Because of the latency you need to pack a small amount of data in VoIP packets so the extra header size of IPv6 stings more than it would for ordinary http traffic
Last time I looked at Digital Ocean they had completely missed the purpose of IPv6 and would only assign a droplet a /124 and even then only as a fixed address like they were worried we are going to run out of addresses.
Because 2^128 is too big to be reasonably filled even if you give a ip address to every grain of sand. 64 bits is good enough for network routing and 64 bits for the host to auto configure an ip address is a bonus feature. The reason why 64 bits is because it large enough for no collisions with picking a ephemeral random number or and it can fit your 48 bit mac address if you want a consistent number.
With a fixed size host identifier compared to a variable size ipv4 host identifier network renumbering becomes easier. If you separate out the host part of the ip address a network operator can change ip ranges by simply replacing the top 64 bits with prefix translation and other computers can still be routed to with the unique bottom 64 bits in the new ip network.
This is what you do if you start with a clean sheet and design a protocol where you don't need to put address scarcity as the first priority.
Yeah, the current system is really weird, with many address assigning services refusing to create smaller pools. I really hope that's fixed one day. We already got an RFC saying effectively "going back to classful ranges was stupid" https://datatracker.ietf.org/doc/html/rfc6177 (for over a decade...)
Point of fact it's giving 4 billion Internets worth of addresses to every local subnet.
You will sometimes see admins complain that IPv6 demands that you allow ICMP (at least the TOOBIG messages) through the firewall because they're worried that people on the internet will start doing pingscans of their network. This is because they do not understand what 2^64 is.
Do the math on 2^64 possible host addresses, multiply by the length of an IPv6 ICMP ECHOREQUEST, and then divide by available bandwidth to determine how long it might take you to scan a single subnet.
Hint: the ICMPv6 packet is no shorter than 48 bytes and there are 1.8446744e+19 addresses to scan.
"Simple" VPS providers like DigitalOcean, etc. really need to get the hell onboard with network virtualization. It's 2026, I don't want to be dealing with individual hosts just being allocated a damned /64 either. Give me a /48, attach it to a virtual network, let me split it into /64's and attach VM's to it - if I want something other than SLACC addresses (or multiple per VM) then I can deal with manually assigning them.
To be fair, the "big" cloud providers can't seem to figure this shit out, either. It's mind boggling, I'm not saying I've gone through the headache of banging out all the configuration to get FRRouting and my RouterOS gear happily doing the EVPN-VXLAN dance; but I'm also not Amazon, Google, or Microsoft...
Do you think anything other than trivial internal networking is a common requirement on DO? I’m not saying it’s not, I really don’t know— I haven’t been in the production end of things for a while and when I was, everyone was basically using AWS et. al. for non-trivial applications. They make it easy enough to set up a private ipv4 subnet to connect your internal services. Does that not satisfy you use case or are you just avoiding tooling that might be obsolete sooner than ipv6?
I use IPv6 on my authoritative DNS servers and that's basically it. To your point keeping it disabled on all my hobby crap keeps everything simple for me. If someone can not reach IPv4 then something is broken on their end.
IMO ipv6 is a perfect example of why interface designers can be valuable on technical projects. One of the genius things about ipv4 is it’s a pre-chunked number you can shout across the room or keep in your head as you run down the hall to your keyboard. IPv6 addresses simply don’t have that feature. If they had kept the 4-chunk format and made it alphanumeric, or added a chunk and made it hexadecimal, or something along those lines, I think they could have reasonably alleviated the problem of running out of addresses while not making the addresses SO unfriendly to remember.
But when designers bring things like that up, you get “it’s really not that complicated,” or “I explained this to my 200 year old grandmother over tea/my 16 month old child over the course of a diaper change/my non-technical wife that I intellectually respect less than I should/etc. and they wrote a book on it the next day,” kind of crap. Human factors engineering. Ergonomics matter in technical products.
NAT doesn't solve everything, and creates a whole new class of problems that you can just avoid by adopting IPv6 natively. And it's definitely not being ignored at larger companies.
In particular, just off the top of my head...
- T-Mobile US doesn't even assign clients an IPv4 address anymore. Their entire network is IPv6 native.
- Many cloud providers charge extra for IPv4 addresses, but give IPv6 addresses out for free.
For trivial cases NAT is easy, for complex situations it's a nightmare. I've been fighting a lonely battle against multiple-NAT VPNs as being the solution to the wrong problem for longer than I care to remember, and I'm tired boss. A few years ago we had a client site go offline because a local network guy just didn't like IPv6 and turned it off, not realizing that a huge amount of stuff was happening automatically and that's why he hadn't been needing to work on it.
Yep, mobile device space ISPs again which is what keeps being argued. IPv6 only connections will never gain full traction outside of the mobile marketplace.
I run dual stack at home with dns64/nat64. I average 50/50 traffic v4/v6. Web browsing gets skewed v6 but large file transfers and some streaming pushed it back to 50/50 overall. My family would revolt if I went v6 only so I'm not sure I'd say its just there for holdouts. Major annoyances include any old device and my hue bridge.
That's a bit like saying AC electricity was just a fancy way of delivering what customers really wanted, DC energy.
I'm sure that DC customers used their Edison DC equipment for decades after the grid went AC only; but in the long run the newer, flexible, lower overhead system became the default for new equipment and the compatibility cludges were abandoned.
Well, yes. Except that AC came to dominance much faster than IPv6, the AC/DC war lasted less than 10 years, with the AC quickly coming to domination. Because AC provides a clear performance advantage over DC.
This is not really true of IPv6. It _still_ has tons of actual operational issues, and in the best case, it does not provide any tangible improvements over IPv4+NAT for the vast majority of users.
For example, in-flight entertainment works by assigning you an IPv4 address and allowlisting it in the gateway rules. This does not work with IPv6 because of privacy addresses and SLAAC. You might think that you just need to do stateful DHCPv6, but Android doesn't support it. Heck, even simple DHCPv6 PD automatic configuration is _still_ not a standard ( https://datatracker.ietf.org/doc/rfc9762/ )!
So to this day, some of the most visited sites like amazon.com, ebay.com, tiktok.com, slack.com or even github.com do not support IPv6. I also keep providing this example, year after year: there are no public VoIP SIP providers in the US that simply _support_ IPv6. Go on, try to find one.
HVDC is enormously expensive even today and completely impractical for bulk transport 100 years ago. You can't look just at corona, capacitive etc. losses of HVAC, you need to factor in the entire economic equation. The total overhead of AC (cost of equipment + energy lost for the lifetime of the line) is still lower for overground transport over reasonable distances and will remain so for the foreseeable future.
I don't think they even had a way to do dc-dc voltage step-up and step-down at high power and efficiency, needed semiconductors for that to do high speed switching in buck and boost converters
No; most sites I reach from the phone seem to be reached via IPv6. E.g. hitting whatismyip.org exposes an IPv6 (though mentions an IPv4 because they're trying to discover that, too).
Some sites do not support IPv6; for those indeed there's a XLAT464 service.
Yes as I said in a sibling post the telcos are the only ones using it, and that is the only reason that graphs like the google client one exist. That is only because it already exists and is cheaper than using NAT when you have hundreds of millions of clients.
> I would say its more "Wireless only ISPs are the only ones using it"
So… the largest ISPs.
Recent number show about 94% of Americans have cell phones and 92% of American households have Internet connections. In raw numbers, that’s about 300M cell phones and 111M households.
If zero fixed ISPs support IPv6, that’d still be about 75% of total Internet connections that do.
What are you even basing that on? Here are some facts:
- You have to pay money to get a static IPv4 address for cloud machines on eg AWS. Anything needing a static IPv4 will cost more and more as demand increases. NAT doesn’t exactly fix that.
- Mainstream IoT protocols have a hard dependency on IPv6 (eg Matter/Thread). Not to mention plenty of 5g deployments.
- Many modern networks quietly use IPv6 internally. I mean routing is simpler without NAT.
So it almost definitely won’t die. It’s more likely it’ll slowly and quietly continue growing behind the scenes, even if consumers are still seeing IPv4 on their home networks.
> IPv4 addresses have been dropping in price for a few years and are cheaper in real terms than at my point in the last 15
More IPv6 deployments may (ironically?) help reduce IPv4 prices as you can get IPv6 'for free' and have Internet connectivity (and not have to worry about exhaustion in any practical way). Doing CG-NAT could reduce the number IPv4 addresses you need to acquire.
IPv4 addresses are basically free - indeed they are a profit centre. At $20 an address that’s $2 a year at most (10% ROI) where many charge 20 times that ($5/month isn’t unheard of)
People love this graph and regularly tout it as if it explains full internet usage. Especially when they dont bother to add any explanation or comment alongside it.
This graph is mainly due to the fact that telcos use IPv6 for mobile devices, nothing more. Over time you will see that graph flatline and peter out as mobile device uage reaches critical mass.
It seems more the other end of the stick: the IPv4 side of the graph is mainly held up due to corporations. The consumer internet continues to switch, but corporate VPNs are going to continue to drag down the numbers until corporations get charged enough for IPv4 address space that bottom lines start to notice.
Yes good point, I agree that IPv4 addresses are going to become a commodity in the future and their value will start to increase dramatically to the point where it is only corporations which can afford to use them. IPv6 use may well start to spike again if that happens.
Yep, and even with all those countries with their billions of mobile devices IPv6 use still hasnt even reached 50%.
Pretty much all ISPs hand out both IPv6 and IPv4 addresses to their clients, this is nothing new. When they start only issueing IPv6 IPs is when it would start truly taking off, but it will never get to that point and it will never happen.
It feels like you are constantly moving goal posts here. Your original statement was it will die a slow and quiet death. Are you now saying that this mobile use case will start to switch back to IPv4? It may not kill IPv4, like was initially planned, but it's not going away.
Apologies maybe slow death was the wrong phrase. I did mean that, but only in the non-mobile space. Obviously mobile device networks have made good use of IPv6 and will continue to.
However In another thread it was argued that when IPv4 addresses become very expensive, that could trigger a big shift to IPv6. I agree with this statement and so IMO it is possible that IPv6 may well become ubiquitous in the future.
Usage is in no way 'rapidly increasing', in fact the google graph everyone is touting around shows that it has taken over 10 years to not even get to 50%. It also shows it is slowing down, the curve is starting to become less steep.
When Maximum possible IPv6 usage is not even at 50% after over a decade and the usage curve is slowing, how can you possibly say that IPv4 is dying and IPv6 usage is rapidly increasing?
Oh, now it’s a problem because it’s been about a decade?
So what, another decade and we should be mostly done?
What do you think is a reasonable amount of time to redo the entire world’s networking infrastructure across 200+ countries and 8 something billion people, exactly?
> Oh, now it’s a problem because it’s been about a decade?
No, but taking over a decade to not even be half adopted does not count as rapid in my opinion.
> So what, another decade and we should be mostly done?
No, as I have said many many times, the graph is slowing.
> What do you think is a reasonable amount of time to redo the entire world’s networking infrastructure
We dont need to, thats the point. All networking equipment in the world already supports IPv6, so why isnt it at 100% usage and IPv4 is turned off already?
>This is an absurd argument, you know that right?
Who is the fool, the person saying what they think or the person continuing to participate in an argument they consider absurd?
You dont need to make everybody in the world agree with what you are saying, it is ok to have differing opinions. You know that right?
I am done now. I accept that you disagree with me and thats fine. Can you afford the same decency or will you continue to tell me I'm wrong?
Jealousy? I think you may need to expand your vocabulary a bit!
I have an opinion on something, I assume thats ok? You seem to be here trying to prove me wrong, and also commenting on my tone of reply.
Im not trying to tell you that you are wrong, only stating what I think. If you dont like my opinion, feel free to ignore it. You are not forced to comment.
Devices maybe, software won't :-\ (We're going to see ever-diminishing pockets of IPv4 around for a loooong time, much like we still see pockets of Cobol.)
> Over time you will see that graph flatline and peter out as mobile device uage reaches critical mass.
...what? The majority of people access the Internet from their phone, and not only since yesterday either. Are you arguing that this is temporary fad somehow?
I am arguing that at some point there wont be any more people without phones, meaning it has reached critical mass and so IPv6 adoption will stall. The number of smartphones in the world will not keep on going up forever.
That would only happen if all of v6's growth is coming from mobile users, no mobile networks are growing/deployed without v6, and also no users are dropping their wired connections.
You can look at the AS breakdowns on APNIC's stats and see that ASs that serve non-mobile customers are getting v6, and that some ASs for mobile users aren't. So no, it won't stall.
Slow down perhaps, but it has to slow down at some point or it'll go above 100%.
Debug symbol size shouldn't be influencing relocation jump distances - debug info has its own ELF section.
Regardless of whether you're FAANG or not, nothing you're running should require an executable with a 2 GB large .text section. If you're bumping into that limit, then your build process likely lacks dead code elimination in the linking step. You should be using LTO for release builds. Even the traditional solution (compile your object files with -ffunction-sections and link with --gc-sections) does a good job of culling dead code at function-level granularity.
Google Chrome ships as a 500 MB binary on my machine, so if you're embedding a web browser, that's how much you need minimum. Now tack on whatever else your application needs and it's easy to see how you can go past 2 GB if you're not careful. (To be clear, I am not making a moral judgment here, I am just saying it's possible to do. Whether it should happen is a different question.)
I just checked Google Chrome Framework on my Mac, it was a little over 400 MB. Although now that I think about it it's probably a universal binary so you can cut that in half?
Google is made of many thousands of individuals. Some experts will be aware of all those, some won't. In my team, many didn't know about those details as they were handled by other builds teams for specific products or entire domains at once.
But since each product in some different domains had to actively enable those optimizations for themselves, they were occasionally forgotten, and I found a few in the app I worked for (but not directly on).
ICF seems like a good one to keep in the box of flags people don't know about because like everything in life it's a tradeoff and keeping that one problematic artifact under 2GiB is pretty much the only non-debatable use case for it.
Oh, there was plenty of internalized propaganda about civil disobedience being the "wrong approach" [0] - it is by no means a new phenomenon. Such criticism was common enough during the civil rights
movement that MLK addressed it in Letter from Birmingham Jail:
> You may well ask: "Why direct action? Why sit ins, marches and so forth? Isn't negotiation a better path?" You are quite right in calling for negotiation. Indeed, this is the very purpose of direct action. Nonviolent direct action seeks to create such a crisis and foster such a tension that a community which has constantly refused to negotiate is forced to confront the issue. It seeks so to dramatize the issue that it can no longer be ignored.
> Oh, there was plenty of internalized propaganda about civil disobedience being the "wrong approach"
Yes, I'm aware of this. But I think it's surprising to me because decades later, MLK et al. are nearly universally accepted to have been right, but people using the same tactics are not.
But the cyclical propaganda lines do cross the decades. In the Bush 2 years I was rather taken by similarities between discussions of the Vietnam war or Watergate (which I read about in books or heard about from boomers) and what was then current events. A lot of the right wing stuff we've encountered more recently reminds me a lot of the 90s, when Rush Limbaugh or Newt Gingrich bursted on the scene. All of the same talking points go in waves. Nothing new under the sun.
The biggest risk to this business model isn't the government, but the payment processor. Anonymity makes it easy for unsavory characters to use stolen credit cards to buy your compute. The inevitable barrage of chargebacks will then cause Stripe to cut you off. Hell, if you're particularly unlucky, your payment processor might even cut you off proactively, if they decide that your lack of KYC makes you a risk.
For any compression algorithm in general, you keep track of A = {uncompressed bytes processed} and B = {compressed bytes processed} while decompressing, and bail out when either of the following occur:
In practice one of the things that happens very often is that you compress a file filled with null bytes. Such files compress extremely well, and would trigger your A/B threshold.
On the other hand, zip bomb described in this blog post relies on decompressing the same data multiple times - so it wouldn't trigger your A/B heuristics necessarily.
Finally, A just means "you can't compress more than X bytes with my file format", right? Not a desirable property to have. If deflate authors had this idea when they designed the algorithm, I bet files larger than "unreasonable" 16MB would be forbidden.
> In practice one of the things that happens very often is that you compress a file filled with null bytes. Such files compress extremely well, and would trigger your A/B threshold.
Sure, if you expect to decompress files with high compression ratios, then you'll want to adjust your knobs accordingly.
> On the other hand, zip bomb described in this blog post relies on decompressing the same data multiple times - so it wouldn't trigger your A/B heuristics necessarily.
If you decompress the same data multiple times, then you increment A multiple times. The accounting still works regardless of whether the data is same or different. Perhaps a better description of A and B in my post would be {number of decompressed bytes written} and {number of compressed bytes read}, respectively.
> Finally, A just means "you can't compress more than X bytes with my file format", right? Not a desirable property to have. If deflate authors had this idea when they designed the algorithm, I bet files larger than "unreasonable" 16MB would be forbidden.
The limitation is imposed by the application, not by the codec itself. The application doing the decompression is supposed to process the input incrementally (in the case of DEFLATE, reading one block at a time and inflating it), updating A and B on each iteration, and aborting if a threshold is violated.
...over the course of 8.5 months, which is way too short for a meaningful result. If their strategy could outperform the S&P 500's 10-year return, they wouldn't be blogging about it.
The reason I really like Zig is because there's finally a language that makes it easy to gracefully handle memory exhaustion at the application level. No more praying that your program isn't unceremoniously killed just for asking for more memory - all allocations are assumed fallible and failures must be handled explicitly. Stack space is not treated like magic - the compiler can reason about its maximum size by examining the call graph, so you can pre-allocate stack space to ensure that stack overflows are guaranteed never to happen.
This first-class representation of memory as a resource is a must for creating robust software in embedded environments, where it's vital to frontload all fallibility by allocating everything needed at start-up, and allow the application freedom to use whatever mechanism appropriate (backpressure, load shedding, etc) to handle excessive resource usage.
> No more praying that your program isn't unceremoniously killed just for asking for more memory - all allocations are assumed fallible and failures must be handled explicitly.
But for operating systems with overcommit, including Linux, you won't ever see the act of allocation fail, which is the whole point. All the language-level ceremony in the world won't save you.
Even on Linux with overcommit you can have allocations fail, in practical scenarios.
You can impose limits per process/cgroup. In server environments it doesn't make sense to run off swap (the perf hit can be so large that everything times out and it's indistinguishable from being offline), so you can set limits proportional to physical RAM, and see processes OOM before the whole system needs to resort to OOMKiller.
Processes that don't fork and don't do clever things with virtual mem don't overcommit much, and large-enough allocations can fail for real, at page mapping time, not when faulting.
Additionally, soft limits like https://lib.rs/cap make it possible to reliably observe OOM in Rust on every OS. This is very useful for limiting memory usage of a process before it becomes a system-wide problem, and a good extra defense in case some unreasonably large allocation sneaks past application-specific limits.
These "impossible" things happen regularly in the services I worked on. The hardest part about handling them has been Rust's libstd sabotaging it and giving up before even trying. Handling of OOM works well enough to be useful where Rust's libstd doesn't get in the way.
I hear this claim on swap all the time, and honestly it doesn't sound convincing. Maybe ten or twenty years ago, but today? CAS latency for DIMM has been going UP, and so is NVMe bandwidth. Depending on memory access patterns, and whether it fits in the NVMe controller's cache (the recent Samsung 9100 model includes 4 GB of DDR4 for cache and prefetch) your application may work just fine.
Swap can be fine on desktops where usage patterns vary a lot, and there are a bunch of idle apps to swap out. It might be fine on a server with light loads or a memory leak that just gets written out somewhere.
What I had in mind was servers scaled to run near maximum capacity of the hardware. When the load exceeds what the server can handle in RAM and starts shoving requests' working memory into swap, you typically won't get higher throughput to catch up with the overload. Swap, even if "fast enough", will slow down your overall throughput when you need it to go faster. This will make requests pile up even more, making more of them go into swap. Even if it doesn't cause a death spiral, it's not an economical way to run servers.
What you really need to do is shed the load before it overwhelms the server, so that each box runs at its maximum throughput, and extra traffic is load-balanced elsewhere, or rejected, or at least queued in some more deliberate and efficient fashion, rather than franticly moving server's working memory back and forth from disk.
You can do this scaling without OOM handling if you have other ways of ensuring limited memory usage or leaving enough headroom for spikes, but OOM handling lets you fly closer to the sun, especially when the RAM cost of requests can be very uneven.
It's almost never the case that memory is uniformly accessed, except for highly artificial loads such as doing inference on a large ML model. If you can stash the "cold" parts of your RAM working set into swap, that's a win and lets you serve more requests out of the same hardware compared to working with no swap. Of course there will always be a load that exceeds what the hardware can provide, but that's true regardless of how much swap you use.
Swap isn't just for when you run out of ram though.
Don't look at swap as more memory on slow / hdds. Look at it as a place the kernel can use if it needs a place to put something temporarily.
This can happen on large memory systems fairly easily when memory gets fragments and something asks for a chunk of memory than can't be allocated because there isn't a large enough contiguous block, so the allocation fails.
I always do a least a couple of GBs now for swap... I won't really miss the storage and that at least gives the kernel a place to re-org/compact memory and keep chugging along.
Sure, but you can do the next best thing, which is to control precisely when and where those allocations occur. Even if the possibility of crashing is unavoidable, there is still huge operational benefit in making it predictable.
Simplest example is to allocate and pin all your resources on startup. If it crashes, it does so immediately and with a clear error message, so the solution is as straightforward as "pass bigger number to --memory flag" or "spec out larger machine".
Overcommit means that the act of memory allocation will not report failure, even when the system is out of memory.
Instead, failure will come at an arbitrary point later, when the program actually attempts to use the aforementioned memory that the system falsely claimed had been allocated.
Allocating all at once on startup doesn't help, because the program can still fail later when it tries to actually access that memory.
FreeBSD and OpenBSD explicitly mention the prefaulting behavior in the mlock(2) manpage. The Linux manpage alludes to it in that you have to explicitly pass the MLOCK_ONFAULT flag to the mlock2() variant of the syscall in order to disable the prefaulting behavior.
I would be suprised if some os detects the page of zeros and removes that allocation until you need it. this seems like a common enough case as to make it worth it when memory is low. I'm not aware of any that do, but it wouldn't be that hard and so seems like someone would try it.
Overcommit only matters if you use the system allocator.
To me, the whole point of Zig's explicit allocator dependency injection design is to make it easy to not use the system allocator, but something more effective.
For example imagine a web server where each request handler gets 1MB, and all allocations a request handler does are just simple "bump allocations" in that 1MB space.
This design has multiple benefits:
- Allocations don't have to synchronize with the global allocator.
- Avoids heap fragmentation.
- No need to deallocate anything, we can just reuse that space for the next request.
- No need to care about ownership -- every object created in the request handler lives only until the handler returns.
- Makes it easy to define an upper bound on memory use and very easy to detect and return an error when it is reached.
In a system like this, you will definitely see allocations fail.
And if overcommit bothers someone, they can allocate all the space they need at startup and call mlock() on it to keep it in memory.
The Rust folks are also working on having local allocators/arenas in the language, or perhaps a generalization of them known as "Storages" that might also interact in non-trivial ways with other work-in-progress features such as safe transmute or placement "new". The whole design space is somewhat in flux, that's why it's not part of stable Rust yet.
I imagine people who care about this sort of thing are happy to disable overcommit, and/or run Zig on embedded or specialized systems where it doesn't exist.
There are far more people running/writing Zig on/for systems with overcommit than not. Most of the hype around Zig come from people not in the embedded world.
If we can produce a substantial volume of software that can cope with allocation failures then the idea of using something than overcommit as the default becomes feasible.
It's not a stretch to imagine that a different namespace might want different semantics e.g. to allow a container to opt out of overcommit.
It is hard to justify the effort required to enable this unless it'll be useful for more than a tiny handful of users who can otherwise afford to run off an in-house fork.
> If we can produce a substantial volume of software that can cope with allocation failures then the idea of using something than overcommit as the default becomes feasible.
Except this won't happen, because "cope with allocation failure" is not something that 99.9% of programs could even hope to do.
Let's say that you're writing a program that allocates. You allocate, and check the result. It's a failure. What do you do? Well, if you have unneeded memory lying around, like a cache, you could attempt to flush it. But I don't know about you, but I don't write programs that randomly cache things in memory manually, and almost nobody else does either. The only things I have in memory are things that are strictly needed for my program's operation. I have nothing unnecessary to evict, so I can't do anything but give up.
The reason that people don't check for allocation failure isn't because they're lazy, it's because they're pragmatic and understand that there's nothing they could reasonably do other than crash in that scenario.
Have you honestly thought about how you could handle the situation better than an crash?
For example, you could finish writing data into files before exiting gracefully with an error. You could (carefully) output to stderr. You could close remote connections. You could terminate the current transaction and return an error code. Etc.
Most programs are still going to terminate eventually, but they can do that a lot more usefully than a segfault from some instruction at a randomized address.
I used to run into allocation limits in opera all the time. Usually what happened was a failure to allocate a big chunk of memory for rendering or image decompression purposes, and if that happens you can give up on rendering the current tab for the moment. It was very resilient to those errors.
I question that. I would expect in most cases that even if you manage to free up some memory you only have a little bit longer to run before something else uses all the memory and you are back to the original out of memory problem but no place to free up more. Not to mention those caches you just cleared should exist for a good reason and so your program is running slower in the mean time.
> If we can produce a substantial volume of software that can cope with allocation failures then the idea of using something than overcommit as the default becomes feasible.
What would "cope" mean? Something like returning an error message like "can't load this image right now"? Such errors are arguably better than crashing the program entirely but still worth avoiding.
I think overcommit exists largely of fork(). In theory a single fork() call doubles the program's memory requirement (and the parent calling it n times in a row (n+1)s the memory requirement). In practice, the OS uses copy-on-write to avoid both this requirement and the expense of copying. Most likely the child won't really touch much of its memory before exit or exec(). Overallocation allows taking advantage of this observation to avoid introducing routine allocation failures after large programs fork().
So if you want to get rid of overallocation, I'd say far more pressing than introducing alloc failure handling paths is ensuring nothing large calls fork(). Fortunately fork() isn't really necessary anymore IMHO. The fork pool concurrency model is largely dead in favor of threading. For spawning child processes with other executables, there's posix_spawn (implemented by glibc with vfork()). So this is achievable.
I imagine there are other programs around that take advantage of overcommit by making huge writable anonymous memory mappings they use sparsely, but I can't name any in particular off the top of my head. Likely they could be changed to use another approach if there were a strong reason for it.
I don't know Zig. The article says "Many people seem confused about why Zig should exist if Rust does already." But I'd ask instead why does Zig exist when C does already? It's just a "better" C? But has the drawback that makes C problematic for development, manual memory management? I think you are better off using a language with a garbage collector, unless your usage really needs manual management, and then you can pick between C, Rust, and Zig (and C++ and a few hundred others, probably.)
yeah, its a better c, but like wouldnt it be nice if c had stadardized fat pointers so that if you move from project to project you don't have to triple check the semantics? for example and like say 50+ "learnings" from 40 years c that are canonized and first class in the language + stdlib
What to say from WG14, when even one of C authors could not make it happen?
Notice how none of them kept involved with WG14, just did their own thing with C in Plan 9, and with Inferno, C was only used for the kernel, with everything else done in Limbo, finalizing by minor contributions to Go's first design.
People that worship UNIX and C, should spend some time learning that the authors moved on, trying to improve the flaws they considered their original work suffered from.
I don't think manual memory management is c's problem. a very large number of errors i see in 'c' programs comes from the null terminated string paradigm and also mistakes from raw pointer manipulation (slices/fat pointers help a lot here).
> Stack space is not treated like magic - the compiler can reason about its maximum size by examining the call graph, so you can pre-allocate stack space to ensure that stack overflows are guaranteed never to happen.
How does that work in the presence of recursion or calls through function pointers?
Recursion: That's easy, don't. At least, not with a call stack. Instead, use a stack container backed by a bounded allocator, and pop->process->push in a loop. What would have been a stack overflow is now an error.OutOfMemory enum that you can catch and handle as desired. All that said, there is a proposal that addresses making recursive functions more friendly to static analysis [0].
Function pointers: Zig has a proposal for restricted function types [1], which can be used to enforce compile-time constraints on the functions that can be assigned to a function pointer.
Linux has overcommit so failing malloc hasnt been a thing for over a decade. Zig is late to the party since it strong arms devs to cater to a scenerio which no longer exists.
On Linux you can turn this off. On some OS's it's off by default. Especially in embedded which is a major area of native coding. If you don't want to handle allocation failures in your app you can abort.
Also malloc can fail even with overcommit, if you accidentally enter an obviously incorrect size like -1.
reply