Thank you Gregory for writing this post. There have been a bunch of announcements about "setup.py has been deprecated", but few have clearly outlined how to move away from setup.py, and more importantly, fewer have outlined what a struggle it is to move away from setup.py.
I was sad to see setuptools officially deprecated, because it looks like another way in which Python packaging is being red-taped away for a non-expert. If someone like the OP (who has 10+ years programming Python) had to do so much for what appears to be a zstd CFFI/Rust wrapper, where does that leave the rest of us?
Here's a python package of mine that uses setup.py: https://github.com/ahgamut/cliquematch/blob/master/setup.py which I have not upgraded to the new tool(s) yet. I think I will need to upgrade it soon. If anyone has suggestions for a tool that will _fully replace_ setup.py, I would like to see tutorials with the following examples:
1. How would I build a package that has pure-Python files and data files? With setuptools I would use maybe MANIFEST.in or package_dir.
2. How would I build a package that has a CPython extension accessed via cffi? (this post points to the answer)
3. How would I build a package that has a CPython extension _without_ cffi, that just wraps some small C code I wrote with CPython's API? What about an extension that uses PyBind11? What about an extension that uses Rust?
4. How would I build a package that requires a "system" package like libblas-dev? Can something like numpy be built optimally without ever writing setup.py? What would a config for that look like? Last I remember numpy used their own patch of distutils to build, I wonder what it is now.
As a TLDR, you have many options 3rd party build tool (aka build backends). Each build tool have different *static* ways to specify compile options that is native to the language or generic (e.g., CMakeList,s Cargo.toml, 3rd party YAML. When it comes to dynamically specifying your extensions, setuptools is still the only option.
I believe the problem is that bridge in the US (not true in some other countries) lacks a "critical mass" of 20 and 30 somethings.
I like to play bridge, but all the people I have asked in my age group (the 20 and 30 somethings) say bridge is a game their (grand)parents (used to) play. I am not sure how bridge can be introduced as a fun hobby without the face-to-face aspect, because compared to other online games it does seem a bit monotone.
I don’t know much about bridge, but ny understanding is that you can legally “cheat” by embedding information in the plays you make. So I guess there is a kind of tension between basic optimal play and communicating effectively with your partner? Sounds like the kind of thing some hackers would really go for.
Bridge is a game of limited information, and making best use of the available limited information and this included signalling WITH card play, but NOT tone of voice, expression, mannerism, or other such "pokerisms".
For instance "When I lead the Ace of a suit, I promise the king" is legal signaling. "When I hold my cards with 3 fingers I have the king of spades" is not.
It's not cheating if you use allowed information bits (eg card value, not the orientation it was placed on the table) and disclose the systems to your opponents. The bidding also has some rules about allowed systems, but it's the same kind of thing. It's a great game for hackers, but the boomers kind of ruin the fun sometimes.
This is the whole point of the bidding system... Except that it's not cheating, because the other team also has a pretty good idea of what you have as well, because people follow the same systems. The author mentions alerts - this is when you have to explicitly call out the meaning of a bid if it's not following the usual convention. Usually you do this as well as announcing what system you're playing at the start whenever you get a new opponent pair. In his article, he gives examples like "1/2 GF" and because that's not a standard system for tournaments, if e.g. the opener played 1S and their partner replied with 2D, the opener has to alert the other team about the non-standard bid by saying "game forcing" immediately after their partner bids the 2D and before the opponent bids.
This is contrary to the normal rules that you're not allowed to say ANYTHING other than a bid which is a number and a suit. That's why it's surprising on the first page that people are getting upset when he's asking them to be quiet. On the other hand, it's quite common to have a friendly chat during the game as long as it's obvious to everyone it has nothing to do with the game. However, that does obviously open up the possibility of secret information being passed as well as being distracting to the other team, so normally people respect the rules and only talk between games. If people are getting violent when being asked to be quiet during bidding, then they should definitely be told to leave by the organisers.
In terms of secret bidding, I have kind of mixed feelings about this. There are definitely opportunities for creating a very complicated bidding system, and over the years, that's how the main systems evolved, but most tournaments require you to play with specific conventions so that both teams understand the bidding.
In general, revealing the rough balance of your hand to the opponents isn't too big a deal, as if one team has a very strong hand, the other team obviously is in a much weaker position and there's little they can do to stop a win. In desperate measures you might bid something knowing you can't win, but might not lose too badly, in order to stop a massive win, but that's not common. However, understanding the other team's bidding lets you figure out the balance of cards in their hands, so you might be able to defend more strategically if you know they're going for a risky slam.
The real advantage of non-standard bidding systems is being able to communicate intent between "I have nothing and can't support", "with what you've got we can easily make game" and "with what you've got there's a good chance of a slam" as quickly as possible so that there's plenty of bidding space left to explore the details of best suit fit / NT, or to figure out that both teams are quite balanced and get out of the bidding while it's still low.
When I was in college you could find a pickup game late at night in the dorms. In grad school we could have two games going every day at lunch and enough skilled players to put together a tournament after finals. Daily play did wonders for my game.
I suspect the issue is less about face to face and more about meeting people to begin with. Meeting people is hard, especially for the very large percentage of us who are neuro-divergent. Getting us to do ANYTHING with you requires that you meet us, so...
...I guess a lesson here would be to be friendly with everyone, including strangers.
"Python 4, but not really", because we want to squeeze out more multithreading performance and be cool again. Some questions from reading the OP:
- How much does performance improve due to this No-GIL thing? Is it greater than 2x? For what workloads?
- Do I have to compile two versions of every extension (gil/nogil)? I would prefer building extensions does not get any more complicated.
- Can I automatically update my code to handle nogil? (a tool like lib2to3 or six)
I had tried to build a bunch of different codebases (git, bash, gcc, curl etc.) before trying busybox, and I'd never seen code like that before. Plus that error came alongside a compiler error in my patch that took a long time to debug, so it was a memorable surprise :)
Would you consider a code pattern like that to be common? What other codebases apart from busybox have it? If there are many examples, I might spend some time trying to update my patch to handle patterns of that kind.
New constants do make compilation a lot easier, but my personal opinion is that the overhead to convert to/from the old constants during runtime is too much.
Every time I used any of these constants, I'd have to load a whole bunch of them into my binary as a large lookup table, and go through that table every time I needed a check in my program. It might not be that slow, but I believe it would definitely be noticeable.
My goal was to make porting easier without changing a lot of source code in either the libc or in the software I was trying to port, and still produce binaries that are close or better in performance. Under those constraints, this gcc patch seemed like the best way to simplify the process.
If I run into enough codebases where SIGHUP is used as an array index initializer, I will probably attempt your suggestion just to measure the tradeoffs. Or you could try it out and let me know if a separate set of constants is better.
> I highly doubt many would be willing to rewrite the world to make some "clever hack" work.
I specifically wanted to avoid rewriting the world in order to port established codebases like git/curl to Cosmopolitan Libc.
My goal with this gcc patch was to answer the question: "what is the minimum amount of source code I would have to manually change in order to port software to Cosmopolitan Libc?". As we find out, sometimes we don't have to change anything to compile code for an Actually Portable Executable -- just do the usual ./configure && make, and a small `objcopy` command at the end to create the APE.
I too thought it was a typo when I caused it to occur while building BLIS, but then I found out it was because I had added some AVX thing to the config that my local computer did not have.
Some compilers (like gcc) emit an 'ud' instruction (https://www.felixcloutier.com/x86/ud) in some situations where the compiler detects undefined behaviour. IIRC this instruction triggers SIGILL on Linux.
I ran into this a few days ago when I was running some else's old code that didn't return a value at the end of a method expecting a return value in C. The program would crash with a core dump and the debugger wasn't very clear either. It's been a while since I've needed to dig into the disassembly view.
I don't understand why the compiler wouldn't just error out if it generates code that will never actually succeed, but I'm sure there's some C programmer's explanation for it.
I'm a C programmer and I don't understand it either to be honest. The compiler obviously knows that it encounters UB in this case because it specifically inserts an ud instruction.
It could at least warn about it (and if it doesn't have the information to generate a useful warning or error at that point where the ud instruction is inserted, then that's obviously a problem that needs fixing).
Dereferencing the runtime symbol with a function sounds interesting! If you can show me an example of where it works, I'd be happy to try it out. I like the if-else-goto arrangement because it fit in perfectly with the other parts of gcc -- if you look at my patch[1], you will find that I had to change very little of gcc's existing code to add this capability.
Also, to add new pure-python packages, you just need to unzip them into a local folder "./Lib", and add that folder to the APE.