Interesting. How did it work getting your photos off of iCloud? Does Apple give you a good way to get an archive of all of your photos? That is, the original quality photos, without manually downloading them individually? (I currently have 446 GB of photos in iCloud…)
Immich iOS app supports backing up photos directly from iCloud in original resolution, with the all EXIF data included. I had 230 GB of photos myself, and I left the phone on the charger overnight with the app running in the foreground and screen locking disabled. In the morning everything was imported.
Some people have instead set Photos app on a Mac to download original photos from the iCloud library and then moved the files directly into the server. I have not personally tried this method though.
> Immich iOS app supports backing up photos directly from iCloud in original resolution
wait that is just crazy!!! Dang my dad is going to flip out when I tell him about this. He's got like 1.5 TB of photos in iCloud and has been searching for a way to get them off. And we're so close to our family storage limit that he gets mad at me when I text him pictures hahaha
There is a community-supported CLI program called immich-go that directly supports reading in iCloud and Google takeout archives, as well as local directories. It works great, and has gobs of import options to set up albums and tags.
[ https://github.com/simulot/immich-go ]
That's the worst service I've ever seen. It asks you the size of each zip file and I said 50G at first. And I couldn't download it because the connection was so unstable. No way to resume it and every 20~30 mins, it failed in the middle. Chrome, firefox, safari were all the same. I tried from a GCE VM as well to see if that's my network problem but didn't help.
I had to request again with 2G and I was able to download files finally. But only one by one. And after download 3~5 files, I had to login again as their login expires so frequently.
I had to do that for days and the download got expired. Oh my god. I had to request it again. And you know what? Their file list wasn't deterministic. I had to download from the beginning. lol
I finally made it and I swear I will never use any cloud service from apple.
Same issue has been going on for me with just about any big download from apple serves. Could be icloud. Could be xcode. Doesn’t matter. It will randomly fail in the transfer and require manual intervention to restart. Been this way for years.
iCloud Photos Downloader isn’t user friendly or pretty, but I finally managed to rip my entire collection without having to install any apple software.
It doesn’t begin at the transmitter either, in the earliest days even the camera was essentially part of the same circuit. Yes, the concept of filming a show and showing the film over the air existed eventually, but before that (and even after that, for live programming) the camera would scan the subject image (actors, etc) line-by-line and down a wire to the transmitter which would send it straight to your TV and into the electron beam.
In fact in order to show a feed of only text/logos/etc in the earlier days, they would literally just point the camera at a physical object (like letters on a paper, etc) and broadcast from the camera directly. There wasn’t really any other way to do it.
Our station had an art department that used a hot press to create text boards that were set on an easel that had a camera pointed at it. By using a black background with white text you could merge the text camera with a camera in the studio and "super-imposed the text into the video feed.
"And if you tell the kids that today, they won't believe it!"
So, it’s interesting. You know how with RAM, it’s a good idea for it to be “fully utilized”, in a sense that anything apps aren’t using should be used for file system cache? And then when apps do need it, the least-recently-used cache can be freed to make room? It’s actually similar for the file system itself!
If macOS is using 153GB for iCloud cache, that’s only a bad thing if it’s not giving it back up automatically if your filesystem starts getting full. Because it means you have local copies of things that live in iCloud, making the general experience faster. In that sense, you want your filesystem to be “fully utilized”. The disk viewer in macOS that shows you filesystem utilization should even be differentiating this sort of cache from “real” utilization… this cache should (if everything is working right) should logically be considered “free space”.
Now of course, if there are bugs where the OS isn’t giving that storage back when you need it, that all goes out the window. And yeah… bugs like these happen too damned often. But I still say, the idea is actually a good one, at least in theory.
What would the alternative be? Simply don't cache anything you get from icloud? Because even if you delete it more eagerly, that's a write cycle.
In fact, avoiding deleting it in case the user gets it again, is going to put fewer write cycles on the SSD, assuming you're going to write it to the SSD at all. The only alternative I can think of is keeping everything from iCloud in RAM, but that is a pretty insane idea. (Also, then the first thing you'd get is people complaining that iCloud eats up all their 5G data caps, etc.)
Of course, but then iCloud might want to cache a reasonable amount of data, say, the 10% the user uses the most. Seeing iCloud caches in the 100+GB arena makes no sense to me, especially if the system isn’t rapidly releasing that storage when needed.
If the ability to release the storage on-demand works correctly (and this is a big if) there’s no reason to limit to 10%. What benefit will that have? If the system works well, deleting the data eagerly accomplishes nothing.
I think the actual system uses filesystem utilization as a form of “disk pressure”, to the point where once it’s above a certain threshold (say, 90% used), it should start evicting least-recently-used data. It doesn’t wait for 100%, because it takes some nonzero amount of time to free the cache. But limiting the cache size arbitrarily doesn’t seem useful.
It gets more complicated when there are multiple caches (maybe some third party apps has their own caches) and you need to prioritize who gets evicted, but it’s still the same thing in theory.
But yeah, if the system isn’t working right and cache isn’t seen as cache, or if it can’t evict it for some reason, then this all goes out the window. I’m only claiming it’s good in theory.
I just want to thank you for taking the time to reply so thoughtfully to someone who is so intent on letting it all go to shit just so they can think themselves enlightened by predicting it.
I have the same response to people who ask me why I don’t leave the country since things are going so bad: fuck that, this is my home. I will always love this country. It is never beyond saving. We have been through worse (the civil war at the very most obvious, but there are plenty of other low points.) We can get through this. We can make it better, we can learn to love our neighbor again, we can learn to trust each other again. We can learn to avoid these tendencies towards hatred. We can’t give up.
So democracy is falling in the most militarily powerful country on earth, citizens are being executed and rights stripped… and you think it’s not OK to be angry?
You think we just shouldn’t discuss it at all because people are angry?
no, it’s perfectly fine to be angry, I would be very concerned with anyone’s mental health who isn’t angry about this.
> You think we just shouldn’t discuss it at all because people are angry?
Well, if we just want to discuss how angry we are, that’s just called venting. That’s fine, vent. But don’t confuse that with discussion. I don’t find venting about how angry something makes you to be all that compelling most of the time. Sometimes someone distills the issue at hand into something very poetic and poignant, and that can sometimes be cathartic, but other than that it’s just pure emotion being tossed around and it just amplifies hatred.
> Putting our heads in the sand will not help.
Not sure where you’re drawing this conclusion that I’m putting my head in the sand. Or that people posting their outrage on HN are somehow not sticking their head in the sand, as if the dispensing of internet hot takes is somehow “doing your part” (hint: it’s not.)
> That’s fine, vent. But don’t confuse that with discussion.
The idea that discussion should be dispassionate and analytical is just wrong. All that does is hides biases.
Discussion should be honest; often that means being messy and angry.
there are a lot of places to be angry on the internet. in fact, basically every single website other than HN is a place to be angry. HN is deliberately not that, or at least it aims to not be that.
But people on HN are angry in non-political threads all the time, to the point that there are several items in the guidelines about it (the latest being "don't be curmudgeonly".)
And not everyone in every political thread is simply expressing anger. The majority of comments in this very thread are reasonable. The ones that aren't have been flagged, which is proper.
But flagging all political threads for "anger," regardless of the actual anger on display, while being far more lenient towards it elsewhere (no one is flagging every thread where someone expresses rage about javascript or AI or the modern web) seems hypocritical.
If HN were only anger, then HN would suck in general, yes. The quality of discussion on this site generally coincides with how much people are able to separate their emotion from the facts at hand. For threads like these that basically drops to zero.
I haven’t really been able to find any comments here that are all that reasonable, other than the meta-discussion we’re having now (and trust me, I hate meta-discussion like this. Honestly I’m regretting even bringing any of this up at this point. I should have just flagged and moved on, and had a discussion with IRL friends or family about it instead. Or talked to a therapist, I dunno.)
It seems to me like you're the one with a strong negative emotional investment in this thread. I don't know what your bar for "reasonable" is but the entire top thread seems fine to me. It's certainly better than many discussions I've seen of LLMs or other controversial but technical subjects.
In any case, I disagree that this thread, much less all "political" threads, deserve to be flagged by default. This community's specific grudge against politics is weird given how much politics gets excused in "technical" contexts.
I have a strong emotional investment in what’s happening to my country. That’s why I’m getting upset. It has fuck all to do with HN. I’m getting upset about what’s happening with my country and predictably taking it out on other HN commenters, and other HN commenters are upset about what’s happening and are clearly taking it out on me. (I basically painted a big target on my forehead saying “I flagged this post” and people are talking to me as if I’m one of the ICE supporters or something.)
I’m not against politics on HN. I’m against anger-driven discussions on the internet in general. It’s not only bad discussion by this site’s standards, it’s bad for the world. As in, the internet causes us to hate each other more than we otherwise would, and those divisions are (I believe) directly responsible for the shape of the political landscape today. This is not a game… people talk online, they develop hatred for other people they wouldn’t otherwise have, and take that hatred to the voting booth.
And I don’t count myself as better than average here either: I’m just as likely to post flippant one-line hot take responses to someone else’s flippant one-line hot take responses. I’m just as angry as all of you. I’m not trying to ignore anything, I’m not trying to silence anyone. I’m only saying that arguing angrily at strangers on the internet is the opposite of constructive towards actually fixing any real problems, and we would be better off with “normal” journalism where we hear the news from experts and discuss it with people we generally trust.
Yes, you should discuss only what is allowed. If you use the technology to dissent against your rulers, then it should be switched off (until you come back to your sense and submit yourself to the mercy of mulas).
> it's easy to see the very real authoritarian bend of comfortable professionals who are smart but also in favor of, say, summary execution of people for protesting
There’s that rhetoric I’m talking about! Thanks for giving a perfect example.
For topics like these, the expectation is that everyone comes in here and expresses sufficient levels of outrage. After all, if you’re scrolling through all the posts showing these awful things, you should have built up the requisite level of outrage by now, so if you post anything other than “HN is obviously ok with executions”, you must be one of them and therefore further evidence that these comfortable professionals are complacent and pro-murder.
The nuanced takes are nowhere to be found, because people who might want to come into these discussions with it, see the rhetoric being tossed around, and think “nope, this is all toxic, no way I’m joining in”, flag it, and move on.
But you can look at that exact situation (people flagging the post) and conclude “yup, the person doing the flagging is okay with executions.” It’s wild.
The sad thing is that there is a nuanced discussion to be had here. In fact it’s critical to this country’s survival that we are able to navigate our way through this. But this discussion, this navigation, needs to happen in small groups, where we can actually engage face-to-face. When we can see each other’s humanity, and know that the other person isn’t a monster, and doesn’t want to see innocent people die any more than you do. Where we can dissect each other’s viewpoints carefully.
None of that is really possible in online forums, because the group think is real, and the rhetoric destroys nuance, destroys compassion, destroys the ability to find common ground. It’s sheer toxicity.
I dunno, I’m still really angry about what I'm seeing. If I had anything to say it’s probably something I’d regret later. I’m talking with my loved ones about it and trying to come to grips with what’s happening to my country. It’s not really time for an internet hot take right now.
Fair enough to be angry, for sure. I am lucky enough to have found a therapist a couple of weeks ago, mostly quit drinking, and ceasing FB doom scrolling.
It's hard.
If it helps, there are plenty of folks doing work. Specifically, get trained by whoever your local rapid response network is. That will put you in contact with actual humans in your local who are in similar situations- for me that has bein invaluable.
You might consider (though perhaps not agree) that no where in my post did I insinuate that "flagging" was the same as taking a position on the subject:
my statement was about why I do, indeed, find utility these conversations you don't find useful.
That a) says nothing about my take on your understanding/ position on the issues around protest or politics and b) is a request to understand my position and not, like, a statement about the morality of your position.
And further, to me "being okay with the summary execution of people for protesting" seems like a pretty specific sentiment, and one which I have heard echoed here quite a bit. I find it super useful to see demonstrated so frequently that a person with excellent technical chops in a domain may often have massive deficiencies in their reasoning, if for no other reason that it helps me understand the weakness of my own cognition.
So, perhaps consider that it's you, in projecting a statement I didn't make in a very short and fairly clear post who is "giving a perfect example" of the level of nuance-free assumptions that do (as you correctly point out) often run rampant- and not just on this site, but in discourse in general.
To push my point a bit further, I am not here to make moral judgements or change peoples' mind on these political topics; rather, over the decade during which I've been interacting on this site, it has= has been super informative to trace the nascent fascism that breed in many of the confluences of technology and capitalism.
That may, to you, sound like hyperbolic rhetoric that is dismissive of other folks' opinions; from my position you're not understanding that this examination of (what is to me) highly disagreeable and almost sociopathic political discourse -is- the process of finding nuanced and useful understandings of our political situation.
The whole thread here started with “why do we continue to ignore this”, to which I replied “who’s ignoring this”, and the answer is “anyone who’s flagging this post.”
The conversation in this particular thread has gone off the rails, in large part because I am very angry about what’s happening, and I tend to get heated in replies. So I apologize for letting my anger get the best of me in this particular instance.
My only point was to say “I flagged this, but not because I’m ignoring it.”
I flagged it because I truly believe to my core that anonymous online discussion about emotional political topics is causing this country’s descent into fascism. Whipping people into a frenzy against one another, causes hatred to amplify past where it would be if it were just about the story itself. The discussions are where people go to out-signal each other, even if nobody’s there to argue the other point. Then if someone does end up saying something like “hmm, looks like the protester was actually carrying a gun” (or something equally not-wrong but clearly not the expected expression of ICE-hatred we all expect), they’re now the target of everyone’s anger. All that brewing hatred is now pointed at that one person, because they’re the closest thing on the site to someone who is actually pro-ICE. Then we have people like you casually saying things about this site being full of tech bros who are just fine with executions… I just feel like we need to tone everything way down. We need to be calm, to be honest. I know it’s hard. I don’t really know what else to say… it’s hard to formulate thoughts clearly in times like these.
I don’t know you but I hope wherever you are you’re safe and have a good idea of what your next steps are going to be. I’m getting increasingly distraught over what’s happening myself. It’s starting to affect my family life and I’m having a lot of trouble coping.
Venting on the internet is a way for a lot of people to come to grips with what they’re seeing. I understand. If this is what helps you cope, I won’t stop you.
Me, it’s especially difficult to see how this hatred is so self-amplifying. I see a president whose primary method of getting to where he is, is to make people hate each other to a maximum degree. I watch liberals like myself fall for it. I see how he intentionally puts armed agents in locations where he knows people will protest, then I see how those protesters are killed in the most predictable way imaginable, because they’re seen as a threat by the people with guns. Then I see the hatred get worse, the protests get larger, with more innocent people joining in, and meanwhile Trump is shipping more armed agents to the same cities.
I wish I had an answer. The answer isn’t “don’t protest”, nor is the answer “let’s all put ourselves in a position to be killed” either. I hope on some level that the images of these people getting executed is seared into enough people’s minds that Trump pays an actual political cost, but then I remember what the BLM protests turned into, and how public support tanked for what should have been an obvious issue. So whatever is going to happen, I’m not sure any of our anger is actually going to help. But I don’t really have an answer. I’m sorry.
My reasons for flagging topics like this is it just fits a pattern of “administration does something abhorrent, people get mad, social media amplifies the anger, it turns into real world deaths.” I really don’t like seeing this happen. I don’t like the hatred amplification that the internet is doing to my country. I don’t know what else to say.
I don't hate you. I'm frustrated with your mental gymnastics. Because your entire thread here boils down to this
> it just fits a pattern of “administration does something abhorrent, people get mad, social media amplifies the anger, it turns into real world deaths
I need you to pay attention to how you're writing. You are laying the deaths of these people squarely on the left for how they react to abhorrence. And if you're going to blame people on the internet for these deaths, you should provide something concrete.
Please outline what bait you think the left took that caused the death of Renee Nicole Good. Explain what "Hatred" was leading us to this moment. Because her last words were "That's fine dude, I'm not mad at you." We must have a fundamental difference of opinion on cause and effect. Because what I'm seeing is a bunch of untrained goons getting frustrated with people and shooting them in anger, knowing that nobody can stop them, and most people making excuses for it.
The idea that the choice is up to us whether we get brutalized is insulting, and frankly, I've seen it being used to excuse political violence for far too long. The idea that the best thing for us now would be to bury these deaths as 6th page news is frustrating, because as someone actually close to this, I've talked with many victims of ICE, and universally, all of them are more upset with the silence and indifference than they are with ICE themselves.
Please, please, please tone it down. Really. This is the amplification I’m talking about. You say “I don’t hate you” and then spew a bunch of hate at me. I’m having a lot of trouble not getting extremely angry right now at your reply because you’re taking the least charitable possible interpretation of every fucking thing I’m saying. This is so fucking maddening. I’m going to reply but please understand this is fucking killing me.
> You are laying the deaths of these people squarely on the left for how they react to abhorrence
> Please outline what bait you think the left took that caused the death of Renee Nicole Good
> Explain what "Hatred" was leading us to this moment.
> The idea that the choice is up to us whether we get brutalized
> The idea that the best thing for us now would be to bury these deaths as 6th page news
I am going to reply to all of this in a chunk. And I want to picture in your mind that I’m someone in your life that you respect (friend, neighbor, loved one, doesn’t matter) talking. Because otherwise I’m just a handle on a forum, and faceless, like the car driving in front of you when you have road rage. It’s so easy to forget we are human beings having a conversation here. Please remember that.
The hate amplification is not causing these deaths. I never said that. I’m saying it leads to them, in a sense that it puts people in the position where it’s going to happen. A death can have nearly infinite things that can lead to it, in the sense that if any one of those things didn’t happen, things could have gone differently.
This is a very important distinction to make. It’s the difference between giving your daughter advice for “don’t dress like that when you go out, there are a lot of dangerous people out there” and saying “it was her fault she was raped”. The former is practical advice you might give to someone you care about, and the latter is the abhorrent victim blaming you’re casually accusing me of doing.
The hatred leads to deaths because there are armed men who are itching to kill liberals, and the hatred makes people think “I ought to go put my car in the way of them.” (I’m not referring to how Renee moved her car in obvious compliance with the officer telling her to, I’m talking about the fact that she had her car there in the first place.) Renee herself probably didn’t have any hatred in her heart at that moment, but I can nearly guarantee she had a lot of conversations just like the ones others are having here, where they look at what’s happening and, in an echo chamber, escalate the rhetoric they’re using. It may go from “ICE is wrong” to “ICE is kidnapping people” to “ICE is the gestapo” to “We need to stop them” to “let’s stop them” to “ok I’ll bring a gun just in case” in the blink of an eye (the gun part being something that’s more relevant to yesterday’s story than the Renee one.)
No, it’s not the left’s fault they’re getting brutalized. It’s the brutalizer’s fault. It’s not fault I’m referring to, it’s “if things would have happened differently this death may have been avoided.” One of those things that could have happened differently is for people to choose a different avenue than directly interacting with the armed men.
No, I don’t have a better alternative than that in mind at the moment, I’m sorry. I know people have to do something, but there has to be something that is less likely to result in deaths than directly interfering with the armed men themselves (yes, filming isn’t interfering, but I dunno, get a zoom lens and maybe do it from farther away. Because if you’re standing right in front of them it very much does seem like interfering. Or in Renee’s case, maybe don’t intentionally park your car in front of the ice agents.)
> Please, please, please tone it down. Really. This is the amplification I’m talking about. You say “I don’t hate you” and then spew a bunch of hate at me
Listen, I'm sorry if I caused you emotional distress, and I can tell you're a kind person, but I wish you led with this comment, because it's a very revealing response. If my comment was "hateful" to you, I don't think you should lecture people on their emotional responses to these situations.
I am going to reiterate that the victims of ICE are begging people to share their stories.
> I am going to reply to all of this in a chunk. And I want to picture in your mind that I’m someone in your life that you respect (friend, neighbor, loved one, doesn’t matter).
I'm talking to you exactly how I talk to anyone in my life that would say what you're saying. Just because I correctly pointed out the impact of your rhetoric, that doesn't mean I'm "spewing hate at you." I'm disappointed in you. I hear a lot worse from people I'm related to.
> This is a very important distinction to make. It’s the difference between giving your daughter advice for “don’t dress like that when you go out, there are a lot of dangerous people out there” and saying “it was her fault she was raped”. The former is practical advice you might give to someone you care about, and the latter is the abhorrent victim blaming you’re casually accusing me of doing.
No man. Open your eyes. You are doing the latter. Renee is dead, and you have 1 sentence condoning ICE, and multiple paragraphs explaining what SHE should have done differently. That is exactly like saying "If she dressed differently, this may not have happened." You have more energy for what she did than the categorically disproportionate escalation of violence.
I wouldn't tell my daughter what to wear, and limit her freedoms, because I know that wouldn't stop her from being harmed. I would tell her to be with people she trusts and lean on her friends for safety.
> One of those things that could have happened differently is for people to choose a different avenue than directly interacting with the armed men
That's great that you're able to do that. Some people do not have that choice. I hope you have a good day.
> No man. Open your eyes. You are doing the latter. Renee is dead, and you have 1 sentence condoning ICE, and multiple paragraphs explaining what SHE should have done differently
I am simply at a loss for words.
This comment thread is probably the least worst in the entire comment section of this post, and it’s still fucking awful. I’m being called someone who’s condoning ice because of my expression that I don’t want to see people die.
Others are being called literal nazis for expressing similar thoughts.
The nuance between “I don’t want people to die” and “I condone ICE” is the exact thing I was saying is getting thrown out.
This is precisely what I predicted would happen from the start, and the exact reason I flagged it. I don’t regret flagging at all, and this is just further proof that I will continue to flag similar threads where people like you lose their empathy and hurl accusations like these around.
If you have loved ones that are victims of ICE and are begging for people to share their story, I don’t think the HN comment section is what they have in mind. Please send it to a proper news outlet.
I wouldn’t say they’re ignoring it’s so much as cheering it on, and falling over each other to voice their support for it. It’s liberals getting killed after all, and they’re not Americans like republicans are Americans.
No, there’s people that love what ICE is doing, people that hate it, people who try and stop it, and the rest of us who look on in horror at the trainwreck and collapse happening in front of us…
But I can’t think of a single group of people who are ignoring it. Other than maybe for a lack of perceived other options and to keep from going insane.
Myself? I’m basically a coward. I have two young children. I don’t want to go protest ICE and get killed by one of these wannabe gestapos. I’m in a real state of fear for my children and the world they’re going to grow up in, but I literally don’t know what else I can do. Maybe I’ll help join the campaign if a democratic candidate this year and help them get elected.
But I’m not ignoring it. I can’t think of anyone who is.
If you're not doing anything about this, you're ignoring it. I don't say this as an insult. I say it as a wakeup call. I'm right there with you, only maybe a few degrees closer to not ignoring it. I have been to protests. I have been tear-gassed and seen people within a few yards of me bloodied up by the authorities, but in comparison to this man that is now dead, I have been ignoring this. For those of us that see that this is wrong, we all need to do more, for your children and mine.
I had the same thing in the house I bought, it was a nice surprise… there were 6 different phone jacks around the house in great locations for Ethernet (WiFi access points or just for a computer), and they all led down to the furnace room where they attached to a punch-down panel (basically they were all spliced into each other.)
To my surprise they were all cat5 cables. With the house being built in 2003 this was surprisingly forward-looking.
I capped all the cables that were on the punchdown panel and put a switch in there instead, and replaced all the wall jacks with RJ45, and bam, working gigabit around the house, including PoE for my WiFi access points. Still haven’t had to punch any holes in the walls.
Same; this was the nicest unexpected surprise about buying this place.
Condo built in 2006 with cat5 . Two bedrooms + living room all wired with rj11 phone jacks. Just snipped those off, wired up rj45, and attached the other ends in my utility closet to a patch panel with rj45 as well.
I don't know if it's just cat5 or 5e, but it saturates a 2.5Gbe link and in-wall cable length is about 15-25 meters.
And you're lucky with that build time, if it was more recent it'd probably be CCA or even CCS. When we redid our place a few years ago I went and bought a drum of plenum cable and told the electricians to use that, so I know what went in there. Overprovisioned slightly but who cares, I had a whole drum of cable and a 48-port switch so may as well use it all.
My mother moved into a retirement village a while back and I was pleasantly surprised to find Ethernet jacks in every room, in some cases more than one. There was no patch panel or anything which was a bit odd, maybe hidden in a service cupboard, but initially I just needed to get a connection from the router to the bedroom and established that these two jacks there were connected. Hooked it up, nothing worked no matter what I did.
On the next visit, with diagnostic gear to look at the wiring map, I found out that the Ethernet jacks were wired up for phone lines. Some genius had decided to run Ethernet to every room, with RJ45 wall sockets, but wired it up for phone lines, so it was simultaneously unusable for either phones or networking.
The only problem with this is that for some god-afwful reason, anything built before the 2010s (?) placed electrical and phone sockets at hip level instead of ankle level. So you're staring at ugly sockets all day.
So sadly you still have to punch holes.
Then again, it isn't that much of a bother if all you have to do is punch a lower hole, relocate the socket and then plaster both holes up and repaint. Especially if you make it a weekend job to do the whole house at once. Or rather, the way I look at it is that it's a weekend job that will improve how the house feels for decades. Doing blind wiring (gutters) for all the ceiling lights falls in the same category.
I think electrical/phone sockets were placed at that level because many telephones were designed to hang on the wall (docking onto and covering up the faceplate) for easy access. My childhood home had one that we used this way before we got a landline.
I was resigned to running cat6e up three floors because there was only coax and I needed a wifi AP up there. Came across the moca solution and it's great. I get flawless 2.5gbe from the basement switch to the third floor over coax. It's basically a little device that connects at each end of the coax and cat6 goes in and out.
Cat 6 would be better though so I could run POE from the basement switch to power the wifi AP, and instead I need to go do a much more complicated switch (cat6) -> moca adapter + power brick to power moca adapter -> coax -> moca adapter + power brick (cat6) -> POE injector (with power brick) -> wifi AP. SO I'm adding at least three power bricks to the setup, which is annoying. Otherwise it would be one cat6 drawing POE from the switch and powering the AP.
You can run power over coax! You can buy power-injecting splitters that were used to power old analog cameras. They basically just connect the cable to the 12V, sometimes directly but usually through some current-limiting safety switch.
MoCA devices have a 100 Ohm internal resistor at the end to limit the cable echoes, so they are not affected by the DC on the cable.
It's worth remembering that UK coax is typically lower quality than that used in the US where these are designed to be used, due to UK coax only needing to transmit terrestrial TV compared to cable in the US.
+1 on MOCA 2 being excellent to solve gaps in wiring. We bought a 6000 sqft 2001 house built with in-wall RJ11, lots of coax runs and some Cat5e runs (but not enough). Due to the size the house, the electrical, HVAC and cabling is roughly divided into two halves with separate electrical panels, HVAC pads, etc.
Unfortunately, all the RJ11 and alarm wiring runs to a closet in one half while all the coax and Cat5e run to a closet in the other half - with no RJ11 endpoints near the Cat5e/Coax closet and not Cat5e/Coax endpoints near the RJ11 closet (sigh). I tried Powerline data and it only works well in adjacent rooms and not at all between the halves due to separate electrical panels. Fortunately, there were a lot of coax runs set up for two separate nets (18-inch satellite and a huge attic antenna for OTA broadcast). So, by repurposing the now-unneeded antenna coax, MOCA 2.5 gbps mostly saved the day by filling in where the Cat5e should have gone but didn't.
My place had previous owners who had the foresight to thread the wire through PVC tube behind the wall. This means that when I wanted to add extra access points, it was easy to thread another cat5 through and pull it to where I wanted.
> It’s also common to find long-running idle queries in PostgreSQL. Configuring timeouts like idle_in_transaction_session_timeout is essential to prevent them from blocking autovacuum.
Idle transactions have been a huge footgun at $DAYJOB… our code base is full of “connect, start a transaction, do work, if successful, commit.” It means you’re consuming a connection slot for all work, even while you’re not using the database, and not releasing it until you’re done. We had to bump the Postgres connection limits by an order of magnitude, multiple times, and before you know it Postgres takes up more RAM than anything else just to support the number of connections we need.
The problem permeated enough of our (rust) codebase that I had to come up with a compile time check that makes sure you’re not awaiting any async functions while a Postgres connection is in your scope. Using the .await keyword on an async function call, but not passing the pg connection to that function, ends up being a nearly perfect proxy for “doing unrelated work while not releasing a connection”. It worked extremely well, the compiler now just straight up tells us where we’re doing it wrong (in 100+ places in fact.)
Actually getting away from that pattern has been the hard part, but we’re almost rid of every place we’re doing it, and I can now run with a 32-connection pool in load testing instead of a 10,000 connection pool and there’s no real slowdowns. (Not that we’d go that low in production but it’s nice to know we can!)
Just decreasing the timeout for idle transactions would have probably been the backup option, but some of the code that holds long transactions is very rarely hit, and it would have taken a lot of testing to eliminate all of it if we didn’t have the static check.
Why don’t you change the order to “do work, if successful, grab a connection from the Postgres connection pool, start a transaction, commit, release the connection to the connection pool”?
That’s what we should do, yes. The problem is that we were just sorta careless with interleaving database calls in with the “work” we were doing. So that function that calls that slow external service, also takes a &PgConnection as an argument, because it wants to bump a timestamp in a table somewhere after the call is complete. Which means you need to already have a connection open to even call that function, etc etc.
If the codebase is large, and full of that kind of pattern (interleaving db writes with other work), the compiler plugin is nice for (a) giving you a TODO list of all the places you’re doing it wrong, and (b) preventing any new code from doing this while you’re fixing all the existing cases.
One idea was to bulk-replace everything so that we pass a reference to the pool itself around, instead of a checked-out connection/transaction, and then we would only use a connection for each query on-demand, but that’s dangerous… some of these functions are doing writes, and you may be relying on transaction rollback behavior if something fails. So if you were doing 3 pieces of “work” with a single db transaction before, and the third one failed, the transaction was getting rolled back for all 3. But if you split that into 3 different short-lived connections, now only the last of the 3 db operations is rolled back. So you can’t just find/replace, you need to go through and consider how to re-order the code so that the database calls happen “logically last”, but are still grouped together into a single transaction as before, to avoid subtle consistency bugs.
We have a similar check in our Haskell codebase, after running into two issues:
1. Nested database transactions could exhaust the transaction pool and deadlock
2. Same as you described with doing eg HTTP during transactions
We now have a compile time guarantee that no IO can be done outside of whitelisted things, like logging or getting the current time. It’s worked great! Definitely a good amount of work though.
I figured it’d be Haskell that is able to do this sort of thing really well. :-D
I had this realization while writing the rustc plugin that this is basically another shade of “function coloring”, but done intentionally. Now I wish I could have a language that lets me intentionally “color” my functions such that certain functions can only be called from certain blessed contexts… not unlike how async functions can only be awaited by other async functions, but for arbitrary domain-specific abstractions, in particular database connections in this case. I want to make it so HTTP calls are “purple”, and any function that gets a database connection is “pink”, and make it so purple can call pink but not vice-versa.
The rule I ended up with in the lint, is basically “if you have a connection in scope, you can only .await a function if you’re passing said connection to that function” (either by reference or by moving it.) It works with rust’s knowledge of lifetimes and drop semantics, so that if you call txn.commit() (which moves the connection out of scope, marking the storage as dead) you’re now free to do unrelated async calls after that line of code. It’s not perfect though… if you wrap the connection in a struct and hold that in your scope, the lint can’t see that you’re holding a connection. Luckily we’re not really doing that anywhere: connections are always passed around explicitly. But even if we did, you can also configure the lint with a list of “connection types” that will trigger the lint.
It sounds super cool, your idea and implementation for await and transactions. Because of my limited Rust knowledge, it's hard for me to understand how difficult it was to implement such a plugin.
Also, your idea of using different domain specific colors is interesting. It might be possible to express this via some kind of effect system. I'm not aware of any popular Rust libraries for that, but it could be worth borrowing some ideas from Scala libraries.
It’s a compile-time check, and yeah it’s a lint rule. In fact it goes a little deeper than a lint can go, because it uses data from earlier compiler phases (in order to get access to what the borrow checker knows.) The correct terminology is a “rustc driver” from what I’ve heard. Lints like clippy run as a “LateLintPass”, which doesn’t have access to certain mir data that is intentionally deleted in earlier phases to lower the memory requirements.
Hopefully it’s something I can open source soon (I may upstream it to the sqlx project, as that is what we’re using for db connections.)
I would love to see how you implemented that (and also the lint itself). I so far haven't found a solid way to implement custom lints for Rust, so if you have any resources to share at some point, I would love to see them!
It feels like Apple lacks the institutional vocabulary to even think about fixing old bugs. The way the releases are structured, there’s a “zero bugs” day where all bugs are ceremonially kicked out of the current release, and the level of quality is deemed to be “what we’re shipping with”. On that day, it’s not like the bugs are fixed, they’re just bulk-modified to target “future os release” and that’s that.
Then the planning is made for next years release and they plan for X features, which require Y time and Z engineers, and some mild hand-waving later a schedule is made, and gee would you look at that, there’s no time anywhere for fixing existing bugs. But that’s ok because big rewrite of subsystem is gonna ship next release and it’ll probably make all the bugs invalid, right? Right? Well, it certainly won’t have more bugs, right? Right? Oops…
> But that’s ok because big rewrite of subsystem is gonna ship next release and it’ll probably make all the bugs invalid, right? Right? Well, it certainly won’t have more bugs, right? Right?
They keep doing them, but I wonder to what degree these rewrites are necessary, and whether your average Apple engineer is aware that they end up with more bugs and vulnerabilities than they started. Surely they've gotta know?
People look at the backlog of issues, see all the things they don't like about $subsystem, and think to themselves "we ought to rewrite this". The incentives are all aligned to make this common. Project managers get a nice chunk of work to manage, engineers get to write things their way, managers get a nice thing to add to their accomplishments, everyone feels like progress is happening. Heck, sometimes there may actually be real deficiencies in the existing code that are being addressed! And in the end, the bug count is lower! (Never mind that it's only lower because it hasn't had the time in production to actually find the bugs yet...)
Large-scale software is hard. So hard nobody's really managed to do it well. By large-scale I don't mean "a lot of users" or "a large deployment"... I mean "a lot of engineers". Once the number of engineers gets large enough, they start making decisions that make the product worse, more bloated, more buggy, and no human is capable of keeping it in check, because the sheer amount of activity in the code is so large you can't possibly keep up with it. And the worst part is that orgs try to solve this by... hiring more engineers to wrangle the complexity. By this point you're already sunk, there's no going back.
Why not have a "rewrite policy", criteria for when a rewrite makes sense or doesn't? Surely a random engineer can't decide to rewrite things on his own.
There’s too many people with incentives to rewrite. It keeps all the gears turning, keeps everyone employed. You certainly need to justify any rewrites, but… people are really good at justifying rewrites.
This seems like it would work if you build a system on solid bedrock, but how often does that really happen? CarPlay, for example, started as a disaster. Unsurprisingly, it has changed a lot but remains one.
Agree that society can survive without it though, but seems a weird thing to just claim as useless.
reply