The front matter of big politics and big tech is not too different from professional wrestling.

Tesla Trailing P/E: 390.70 Apple Trailing P/E: 34.11 Ford Trailing P/E : 11.74

A few years went by and suddenly playing MUD / MOOs was free. I honestly miss those days, text-based has a vibe that no graphical game can ever replicate.

There was a small regular community which got wiped out when WorldPlay started charging for it. The studio got more free accounts and gave them to regular players so that the game didn't just evaporate immediately, which meant everyone suddenly had a minor name change. After that, the game evaporated.

Had a great business idea just now: A tool for staged interviews! The subject and the journalist submit an equal length list of questions. Each round of the auction they bid on questions they want to include or exclude. The loser gets 50% of the points spend by the winner to be used in the next round. Both the subject and the journalists can buy additional points at any time. I keep all the money.

I'm not really intending to single out this essay for this, it's just another in an increasing stream of similar essays. I find it odd because I'm hard-pressed to think of another tech that devs had to be strong-armed into using. In my experience, when devs find value in a thing, they just start using it naturally.

Spotify is awful for me, I concur with the original article. YouTube Music is heading slowly the same way. At this rate I'll have to cast around for another phone that can take an SD card again.

The service I really miss is eMusic, they had little in the way of well known music so leaned into small label music and it was wonderful.

- A section with 8 of my recently played playlists

- A section of "Made for <my name>" with 6 Daily Mixes (which I generally like), Discover weekly (which I like now that it's tailored to me: I used to hate that it only contained pop/hip hop hits), Release Radar (love it), and the AI DJ (which I find very annoying)

- A section called Recently Played which looks like all legitimate things I've played

- "New Releases for You", which are all by artists I've listened to very recently

- "Jump back in", which has several playlists and artists I've listened to recently

- A sidebar of all of my playlists I've created or followed

Of the ~50 actionable items on the page, the only one I dislike is the AI DJ, but it by no means feels forced on me since it's just a single square.

https://i.imgur.com/RKsIgLR.jpeg

The last time I used Spotify (and the reason I left) podcast were constantly featured on the start page and it was impossible to remove them (and I have zero interest in podcast in my music application).

Over time, I have reflected and shifted this perspective. That reflection centered around the question the author hasn’t asked himself yet: who creates these expectations? It’s always some loose definition like “society” or “elite”, or some other handy wavy grouping of people.

But, to be clear, those expectations are coming from the author. That’s his perspective of what the world thinks, based on his interactions, based on what he chooses to read, listen to, etc. He's not describing anyone but himself.

I found that if I find myself projecting onto social norms, society, or some other loose definition of “they”, what I’m really doing is projecting a part of myself that I haven’t been honest about yet.

Money is a huge stressor in relationships. Misogyny is still a huge thing (to my utter surprise honestly). Abuse, etc. Options for financial stability to escape these things I would suspect to contribute as well.

https://en.m.wikipedia.org/wiki/Bowling_Alone

https://www.goodreads.com/book/show/22609334-our-kids

https://www.goodreads.com/book/show/50124402-the-upswing

One thing I want to mention here is that it seems that social effects come before economic effects. People go through social change and then the economics follow later. That’s the one thing I would take away from the books/talks. Somewhat ironic to many here.

This belief is oddly common, considering it has no basis in reality. The share of adults living in households with sub-middle-class incomes has only grown by 4% since 1971.[0] The share of adults living in upper-class households grew by more (5%)! And this even as the number of single-adult households has soared[1], reducing the average number of earners per household!

[0]https://www.pewresearch.org/fact-tank/2018/09/06/the-america... [1]https://www.pewresearch.org/social-trends/2011/12/14/barely-...

1 - https://www.pewresearch.org/ft_dual-income-households-1960-2... 2 - https://www.pewresearch.org/fact-tank/2018/08/07/for-most-us...

That “flat” wage is accounting for inflation according to your own link. So not only would a single earner be doing just as good today as back then, adding a second income means they are raking in twice the spending power as back then.

The only thing that has changed are expectations. People want way bigger houses, nicer cars, computers, smart phones, more meat, dental care, better medical care, etc. Worst of all (from a financial perspective), they want both of their average kids to spend 4 years at an expensive university.

Expensive only because it's been allowed to become expensive. Decades ago it was normal for a good university to be nearly free or at least quite affordably by the student working part time to pay their way.

> dental care, better medical care

People "want" health care. I mean of course, but it's not a whim, it's a basic necessity. Which like the university, was very affordable to nearly everyone decades ago.

Not sure people want bigger houses either, every housing topic on HN is full of desire for smaller apartments to get built.

Here's a few graphs that show this more explicitly: https://www.buzzfeednews.com/article/gregschoofs/how-much-co...

Your data only represents "married couples with children under 18"--a (shrinking) minority of households. By definition it does not account for the dramatic rise in single-adult households.

Less individual freedom, greater direct economic dependency on others in your group, and also the threat of ostracization, excommunication, and destitution if you go against its rules. Basically, there are extremely high cost/stakes associated with going it on your own.

But on the other side there are also good things that come from greater direct interdependency, like perhaps less individual alienation and a greater sense of shared purpose, and access to community resources when you follow the prescribed rules.

E.g. money gets shielded by community institutions (such as local religious organizations) that are harder for elites to steal from than individual peasants, and then those act as a kind of social safety net in hard times. Extended families/clans build social bonds through e.g. marriage and baptism, and help each-other.

Often there are severe social problems in rural peasant societies: alcoholism, domestic violence, seasonal migrant labor keeping people away from home much of the year, corruption, ..., but people have also learned to be tough vs. some kinds of outside threats. But large waves in the world economy (or a large natural disaster or the like) also can overwhelm those defenses.

However these come at the expense of many liberties (i.e. religious) that are held as important in the more developed world. It's not clear that loss of such personal liberties would be an improvement in developed societies, even if it reduced the nominal divorce rate.

When there is a trade off, who should get to make the decisions about where to strike the balance?

A significant proportion of people in the US seem to pine for the days when homosexuality was taboo and illegal, women could be beaten or raped by their husbands (and children by their fathers) and it was treated as no one else’s business, non-white people were kept out of the neighborhood and interracial marriage was frowned on if not illegal, pre-marital sex was encouraged for men but made women into “sluts”, middle/high school students received no education about basic human biology/anatomy, rape victims were forced to deliver their rapists’ babies, most professional jobs were reserved for white men, etc. But hey, low divorce rates!

New normal can easily be pervasive dysfunction. When dysfunction becomes pervasive, it is also perceived as normal by people inside that society.

> The notion that social anomie, abuse and violence is a simply unescapable "fact" about late modern societies is baseless.

These were not so much anomies as taboo to talk about if it is happening to you. They were seen as private issues that should have stayed private and if you did talked about it, you was the bad one. Nevertheless, some statistics are available - for example domestic murders. Those went down. Some anonymous statistics. People who went through it in the past and did talked about it, their children remembering and talking about it later. The way domestic violence is portrayed in media - whether it is shown as something justifiable and ok or not.

When I was child, there was no domestic violence around me. Then I grew into adult and people started to talk more openly in front of me. Turned out, there was in fact domestic violence among adults I knew as child ... except I was protected from it.

Whether it qualified as "epidemic" I don't know.

Seriously, people of all classes were also willfully blind to sexual abuse for years and that includes abuse by priests.

Not really. This was a big part of why people were so vicious during the Protestant Reformation. You don't just have people buried up to their necks and then trampled by horses because you disagree about points of scripture.

But (b) sure this also applies to upper middle class people across “the West” as of not very long ago. It’s not that every household was full of abusers, but it was treated by the public as a private matter, not talked about, and much more widespread than publicly recognized.

Public opinion tells us little about real-world behavior. The whole point of OP is that the upper-middle class liberal elites are not practicing what they preach to the rubes and proles.

My claim is that “low divorce rates” historically often masked widespread abuse and unhappiness in stressed (even broken) marriages which were continued due to social pressure, not always for the best.

* * *

As for the article, this causal claim is wildly speculative bullshit:

"The educated class decides cohabiting partnerships are just as valid and important as marriage. And they also believe it’s okay to walk away at a moment’s notice from a cohabiting relationship. ¶ Poor and working-class people follow suit. To the detriment of themselves and their children."

The problem working-class people have is not bad “elite” role models, but a lack of money and good stable jobs, limited parental leave, a lack of cheap childcare options, a corrupt and exploitative criminal justice system, etc.

The supporting evidence presented in TFA is some papers about how people find newspaper op-eds persuasive, are impressed by qualifications when reading public policy recommendations, and choose their high heel shoe height based on local trends when moving to a rich neighborhood; extrapolating from this to young working-class parents separating from their partners because “elites” say it is okay is a ridiculous stretch. Especially when the young women directly quoted said clearly why they broke up. Occam’s razor says we should listen to what they say instead of inventing some secret reason without any direct evidence.

> When there is a trade off, who should get to make the decisions about where to strike the balance?

Of course the Supreme Court when concerning anything involving the government making an establishment of any religion.

The US isn't Saudi Arabia. As much as it's any American's right to practice the religion of their choosing, it's not in any religion's right to deny any individual - even in their religion - their individual liberties, or to impose their religion upon a person of another or no religion.

There is nothing wrong with religion as a framework for teaching people about their social obligations, but it's hardly the only way to achieve that.

For so-called poorer countries, they are differently structured. It's tough to compare their family units with our rich nations.

But that’s exactly the point! Prior to the 1960s revolution in social norms, families in poor countries weren’t structured all that differently than ones in the US.

I’m no Reaganite, but the fact is that the stuff the old “family values” conservatives said is pretty much the same thing my Asian immigrant parents told me growing up.

Meanwhile, the data shows that Asian Americans who grow up in the bottom 20% have a 25% chance of ending up in the top 20% as adults. For white kids it’s just 11%. That’s a really big coincidence to hand wave away.

The crazy successful asian immigrants are mostly those who came recently, and who are bluntly the best and brightest of Asia coming to America for a better life due to US policy attracting the very best. Just like the above conversation, you only think of one slice of data to arrive at your conclusion.

Men who don’t feel connected to a clear path of ascension become desperate and often either dangerous or losers.

It’ll be interesting to see what happens to the increasing number of men who feel they have “fallen off the path”. Could be scary

They will probably seek revenge against the social norms and classes which alienated them. History tends to repeat itself.

You can make a fair argument that the bottom 50% haven’t gotten their share of productivity growth, but that doesn’t explain why social indicators among the bottom quantile have gotten so much worse.

It's grown for every group used in that breakout, but while it breaks out the upper income groups on relatively fine categories, it groups the bottom half as one category.

(It's useful for the graph's original purpose of discussing top-weighted inequality, it's not useful for the argument you are trying to make with it.)

And education, housing, and health costs has sky-rocketed, number of "essential services" to pay for (mobile phone, internet, severan "rentier" subscriptions) has increased, while job stability and availability for working and middle class has plummeted...

But yes, the top 10%, the types to usually post at HN, never had it better.

I agree the top 10% have poor insight into what life is like for everyone else, but that cuts both ways. I think there’s a real misperception of what income is necessary to maintain a similar quality of life to the past.

Because Germany doesn't treat non-college graduates working class people as "losers".

Germany also has a quite solid social protection, not a cut throat environment where tons of people are a medical bill away from homeless.

Neither does the United States? Not sure where you're getting this idea from. Most people I know revere trades as they're a known way to make great money without a degree. And, anecdotally speaking, I've managed to get great jobs as a software engineer without a degree. We treat non-college graduates who don't try to ascend above retail or fast-food as losers - and for good reason.

What?! The labor market has been so strong over the past decade that economists started to wonder if previous paradigms no longer apply (i.e. Phillips curve).

How does the gutting of the middle class harm stability of working class families?

yes, this is why I couldn't quite be convinced of the thesis being proposed here. Or at the very least, they way they try to describe the "marketing" of being in a faithful marriage runs counter to the reality. While the situation of "deadbeat dad becomes alcoholic, won't get jobs, and cheats" is not an uncommon cause of divorce, the more common narrative pushes it as a liberation of women's right. Less about a man breaking away to be free and more about a woman not being trapped.

That above deadbeat stereotype has more become the "violent domestic abuser" type as of late. So divorce is a way to turn an absolute nightmare of a situation into emotional stability in exchange for what will likely be financial hardship. It turns a bad into slightly less bad, not a potentially patchful good into a bad.

Or at least, that's how society markets it. I don't have hard sources and admit these are just the notions I feel have risen over 3 decades of media consumption.

> It used to be high-status to hold a job and take care of your family. Not so much anymore.

As though it’s ever been considered high-status to have crap pay and deal with abusive bosses and coworkers. Ask the really high-status people whether it was.

You need recordings to be able to review interactions for who is more in the right, telling the employee they need to turn off FB and get on with work seems like it management would say it’s reasonable and worker might say it’s overbearing.

You know what I saw in rich kids?

Lots of drug use, stealing, petty vandalism and violence.

I just rarely saw them suffer any consequences for it.

"Crime" is a function of poverty in so much as those punished for committing it are poor.

People don't need money, but they desperately need sense of agency and entertainment and lack of money makes fulfilling those core needs in legal and moral manner super hard.

You will find across many eras and cultures in the last two millennia that the poorest members of the society are the most vulnerable members of society and the most likely to be punished for committing crimes. If you still don't believe me, read this paper for an in-depth analysis that controls for many factors: https://web.worldbank.org/archive/website01241/WEB/IMAGES/IN...

Laws are made by those with power. In today's society, power comes in large part from wealth - that's the foundation of capitalism. The objective of capitalism is to accrue capital, and our laws and police system are set up to protect the wealthy and their wealth.

That's why graffiti is punishable by ten years in prison. Stealing a week's worth of groceries can mean years in prison. In contrast, the penalty for illegally evicting a renter, rendering them homeless, is about two or three months rent, and no prison time.

Not even close to accurate, unless you're running off with a cartful of steaks, which might push you into criminal territory. Most shoplifting is not even a misdemeanor, just a civil infractions. Criminal charges require hundreds or thousands of dollars and usually multiple offenses to get more serious than probation.

I imagine this explains the correlation between poor minority neighborhoods during the crack epidemic or between the depressed rural areas and the opioid epidemic now.

I suggest reading the book "Manufacturing Consent" to get an understanding of how expectations are shaped by mass media.

Just like there was a concerted effort to create public support for war (WW1/WW2/Vietnam/etc.), there are concerted efforts to shape public opinion and create expectations within our broader culture about morals/values/etc. Sometimes it's explicit, and often times it's implicit and picked up via signaling cues (i.e. high status reporter says X, to go against X would mean no more invites to fancy dinner parties)

The last conclusion is a little off. Everybody who expresses any opinion is describing "his perspective of what the world thinks, based on his interactions, based on what he chooses to read, listen to, etc". That doesn't mean they can't also coincide with a general trend, for example "societal expectations" at large.

The glaring piece missing from this analysis is the psychology of the young men themselves.

Raising two young men myself, I’m blown away by how powerful cultivating intrinsic motivation is. My eldest just went from constant gaming/sleeping/hiding in his room to working a full-time job, launching a side hustle, and enrolling in classes.

Yes, my modeling & encouragement mattered. But the fulcrum was him accepting that living up to other people’s expectation in high school had left him tired, depressed, and lost.

The author has clearly identified that young man lack a goal worthy of their efforts.

He’s just completely ignored intrinsic motivation and the power of cultivating it.

I don’t think that’s an accident. I think our entire approach to young men as a society ignores the fundamental power of their interests.

For me it felt like the title would reflect reality better if it was rephrased to "Society has nothing to offer to young men and they respond by not pursuing anything."

With that said, the tasks are going to come on hot and heavy sooner than you expect. I didn't have homework in grade school. At all. My kids had mountains of homework. Giving them too much to manage on their own, while telling you not to manage it for them, would seem like a cruel joke, but I don't know any other answer.

Also, you could be more disciplined than I was about my own chores. I didn't need to clean the bathrooms every week at the same time, but maybe had I set up a schedule like that, it might have modeled better habits in my kids. Or maybe not. ;-)

I think effective social commentary is mostly grounded in the personal, the specific, and the (thoughtfully) statistical. Otherwise it's very easy just to project one's biases onto a complicated picture. E.g., consider whatever it is that people think is "ruining the kids" these days. In the 1700s, you know what was ruining the kids? Novels: https://www.economist.com/1843/2020/01/20/an-18th-century-mo...

It's similar to how arguments about "what God wants" go wrong. Researchers showed that when people talk about what their god wants, they're unconsciously consulting their own preferences: https://www.discovermagazine.com/planet-earth/creating-god-i...

With broad social argument, it's easy to introduce all sorts of fallacy. Humans aren't really equipped to reason intuitively about things at this scale. Especially if the point is to persuade and/or entertain, simplifying into broad "logical" arguments can paint false pictures. Andrew Gelman wrote about this today: https://statmodeling.stat.columbia.edu/2022/04/24/the-perils...

Honestly, I don't think people are "better" at that (if such a fuzzy construction can have a useful meaning). But I do think they are more careful with it because it's harder to get right and more easy to visibly get it wrong. It's fine with me if expecting more concreteness just drastically reduces the total amount of social commentary. Even if it dropped by 99%, we'd still have more than we needed.

He doesn't provide any data to reflect what society's expectations are. Those are just his thoughts.

That's a big distinction.

In a fascinating 2012 paper titled Sexual Economics, Culture, Men, and Modern Sexual Trends, the psychologists Roy Baumeister and Kathleen Vohs wrote:

“Although this may be considered an unflattering characterization…we have found no evidence to contradict the basic general principle that men will do whatever is required in order to obtain sex, and perhaps not a great deal more. (One of us characterized this in a previous work as, ‘If women would stop sleeping with jerks, men would stop being jerks.’) If in order to obtain sex men must become pillars of the community, or lie, or amass riches by fair means or foul, or be romantic or funny, then many men will do precisely that.”

Especially the quote where an agreement that stability is important was interpreted as agreement that marriage is important.

Rayiner: have you looked at the demographic pyramid in South Korea, Singapore, Japan, Taiwan, or urban China recently? https://upload.wikimedia.org/wikipedia/commons/thumb/4/4a/So...

Or if we want to talk about places where “traditional family values” (i.e. opposition to women’s rights) poll very high, how about the demographics of Russia?

The demographic decline in “a handful of European countries” is substantially a matter of time since industrialization, access to birth control, urbanization, female literacy, amount of immigration, housing prices, etc. more than ethnic/cultural/religious origin, and it is going to land on everywhere else in the world (USA, South Asia, the Middle East, Latin America, Africa, ..) soon enough.

https://en.wikipedia.org/wiki/Argumentum_ad_populum

https://en.wikipedia.org/wiki/Truth_by_consensus

I googled "stability of cohabiting couples". This led me to Wikipedia [1]. From there I hit the 2002 CDC report "Cohabitation, Marriage, Divorce, and Remarriage in the United States" [2]. That provides the following statistic:

"The probability of a first marriage ending in separation or divorce within 5 years is 20 percent, but the probability of a premarital cohabitation breaking up within 5 years is 49 percent. After 10 years, the probability of a first marriage ending is 33 percent, compared with 62 percent for cohabitations."

I wondered if that had changed since 2002, so I hit Google Scholar and searched for articles after 2010. A 2020 article discussed cohabitation broadly. I scrolled down and found a 2018 article [3]. (There's an ungated PDF elsewhere if you need it.) They look at 8 countries including the US. Their findings: "cohabiting couples who do not subsequently transition to marriage... consistently have the highest predicted probabilities of separation within 5 years." Some of the gaps between cohabiting and married couples disappear if you control for (e.g.) education and other demographics.

[1] https://en.wikipedia.org/wiki/Cohabitation#Likelihood_of_spl... [2] https://www.cdc.gov/nchs/pressroom/02news/div_mar_cohab.htm [3] https://read.dukeupress.edu/demography/article/55/4/1389/167...

I heavily dislike the illusionary idealism from some for something that can be quite a lot of work. The state of marriage is pretty much not relevant though. At least in my opinion.

The average human being has 1.99 eyes.

Why can't broader family structures be considered, and be stable?

No one knows how other models work. However, note that "stable" comes from the data. Maybe other models are stable, maybe they're not. Without data, it's simply unknown.

Stable home environment = GOOD biological parent until child becomes adult (if other GOOD parent dies, not leaves)

Leaves = child trauma, parent BAD = child destructive behaviour

One GOOD parent dies = child trauma, but lesser extent

GOOD = morally, respect, supportive, nice, kind, good-feel stuff

/eli5

Lesson: Marry not for looks, but personality eg. Johnny Depp and Amber Heard

I'm not at all sexually frustrated like most men and yet I still feel all of the same things.

I mean clearly psychology is a lot of "our area of study observes X and attributes Y as a part of theory Z". It's not a hard science where objective, absolute, undeniable measurements can be made.

But it's not some random dude just pontificating about social structures on the internet.

By your measure we should just throw all the great philosophers works in the toilet because "that's their perspective".

That's a heck of a false dichotomy. People are either billionaires or working at mcdonalds now?

My opinion is that Dart's type system is the optimal type system. It allows for statically typed API surface, but at the same time, I can still add some dynamic voodoo under the surface and enable productive meta-programming.

    > As a far as I can tell, an "optional type system" is
    > just an unsound type system with a marketing name.

Dart 1.0 was an optionally typed language. Dart 2.0 is not -- we've gone to a mandatory static type system (with lots of inference and a dynamic type). The limitations of optional typing are simply too onerous, and it's just not what users expected or want.

    > Any decent static type system would allow one to 
    > progressively increase the level of typing, the weakest
    > form being using a Variant type to hold all other types.

This would be solved in a conventional static type system by forcing the user to eliminate the variant, and forcing the user to deal with the case that it has an unexpected value.

So I cannot see why even C# dynamic is necessary, except to perhaps save some thinking, but this isn't a good thing IMHO.

Makes little difference, either way you need a magical type since it has to allow any method call at compile time, and then requires enough RTTI that it can check for and dispatch method calls at runtime. Whether you call it Variant or dynamic has no relevance.

The semantic differences for OLE2 Variants vs C# dynamic mostly only exist in implementation protocols.

> The problem munificent was talking about is that for more complex types you cannot write that type checking predicate you want, which would check if a value belongs to the given type and return a variant-free version of the object.

The complexity of the type should not matter to a sound type system. Assume a "fromVariant" function that unpacks the variant, it has a type: forall a. Variant -> Maybe a. Maybe is either Nothing or Just a, depending on whether the variant does indeed contain what you expect it to contain.

> if you have a reference to an array of integers in the typed part of the code you need to protect against untyped parts of the code inserting non-integers inside the array.

This "protection" is exactly what static type systems do. "Untyped code" in the context of using a static type system, is code that works with variants. So we need to pack the array of integers into a variant, where they will be suitably tagged by the runtime. The "untyped code" is free to write whatever it wants into a variant, so we must unpack it with runtime checks. The static type system forces us to unpack and assert its contents, before we can assign a specific type to it.

> The only solution is to insert typechecks in the reads and/or writes to the array

I am not sure what you mean here. When the array is packed into a variant then it can indeed be modified into anything by "untyped" code. But it should always be tagged correctly, which is the responsibility of the runtime. It will need to be re-asserted when unpacked into a specific type.

> A similar problem happens with functions

Functions should be values like any other, so my points apply to them too.

The point I have been trying to make, is that a conventional static type system will already allow for "gradual typing". I cannot see a need to use an unsound type system in order to achieve this goal.

Anyway, going back to the topic, I'll try to explain why your idea doesn't work.

> assume a fromVariant function

What I was trying to say on my other post is that you can only create such a function for primitive types (numbers, booleans, strings, etc). And perhaps also for objects, if your language has something analogoud to an instanceof operator.

But for the higher order stuff like functions and mutable references you cannot write such a fromVariant function so easily! Suppose I write down a dynamically typed implementation of the identity function and pass it to some typed code that expects an int->int function. When we pass the function to the typed side we will call fromVariant on it but the best we can do at first is check if the value we have is a function (and not an integer, etc). There is no way that fromVariant can also check the function signature, to guarantee that the dynamically typed implementation will always return an integer when given an integer.

One way to solve this problem is for fromVariant to return a wrapped version of the dynamic function that checks if the returned value is of the expected type. But these wrappers can slow down the program a lot and they also mess up with pointer equality...

> the arrays

The basic problem with the arrays is similar to the one with function. The statically typed code can't blindly trust that the array will always contain the type it expect because at any moment the untyped portion of the code could try to insert junk values into the array. You will either need to do a runtime check when the dynamic code writes to the array (which slows down the dynamic code) or you need to add runtime checks when the static code reads from the array (which makes it slower than it would be in a fully static language).

Things also get extra tricky if the array contains non primitive values (functions, other arrays, etc)

---

If you are interested I can send you links to a couple of gradual typing papers. As munificent said, there is currently lots of ongoing research about this and we all wish it were as simple as you were suggesting it would be :)

I was trying to show that "gradual typing" should be possible in any reasonable static type system, without resorting to unsoundness. I do concede that there is probably a definition of the term "gradual typing" out there that I am not strictly adhering to.

> Suppose I write down a dynamically typed implementation of the identity function and pass it to some typed code that expects an int->int function. When we pass the function to the typed side we will call fromVariant on it but the best we can do at first is check if the value we have is a function (and not an integer, etc).

If the best we can do is tag it as a function, then fromVariant would unpack it to a function with type Variant -> Variant, which represents all dynamically-typed functions. When we call this function, we indeed would need need to assert the outputs.

> One way to solve this problem is for fromVariant to return a wrapped version of the dynamic function that checks if the returned value is of the expected type

Agreed. Although the danger of this is that you have a bomb waiting to go off inside some unsuspecting piece of typed code. The alternative to to acknowledge that it is a partial function in the type, e.g. the Variant -> Variant function becomes Int -> Maybe Int, or similar. The point is that we are solving these problems using conventional static typing. You haven't yet convinced me that this doesn't work.

> You will either need to do a runtime check when the dynamic code writes to the array (which slows down the dynamic code) or you need to add runtime checks when the static code reads from the array (which makes it slower than it would be in a fully static language).

It sounds like you are concerned with performance and efficiency, that may indeed be an issue, but it's orthogonal to the type system. My own position is that if one wants performance, then don't write untyped code!

> If you are interested I can send you links to a couple of gradual typing papers. As munificent said, there is currently lots of ongoing research about this and we all wish it were as simple as you were suggesting it would be

Thanks for the offer, I am definitely interested. I suspect that the outstanding issues are around usability and efficiency. I stand by my original point that "gradual typing" should be possible in any sufficiently expressive static type system.

The basic goal of gradual typing is that you can have your program be fully untyped or fully typed or somewhere in between and it will behave in a sensibly in all cases. Adding types to a program does not change the result of the program, except that sometimes adding types may cause new type errors to show up.

> the best we can do is tag it as a function, with type Variant -> Variant

This is very limiting. With this restriction, you cannot assign an accurate static type to a function with a dynamically typed implementation, which means that you can't use dynamically typed libraries from inside statically typed code.

Similarly, you cannot use statically typed objects inside the dynamically typed code because all objects that can be touched by the dynamically typed part of the code must have all their fields typed with the Variant type.

> Although the danger of this is that you have a bomb waiting to go off inside some unsuspecting piece of typed code

Technically the bomb is inside the dynamically typed part of the code :) One of the nice things you can get with a well though out gradual typing system is a guarantee that the line number in any of the type error messages will point to the dynamically typed part of the code. We call this blame tracking.

> It sounds like you are concerned with performance and efficiency, that may indeed be an issue, but it's orthogonal to the type system

Yeah, but one of the big challenges with gradual typing today is that i we try to be flexible (letting programmers add types wherever they want) and sound (with a soud type system, blame tracking, etc) all the implementations so far have had very bad performance, often running much much slower than the untyped version of the code would (with factors like 5x or 10x slower being common).

When the performance gets this bad it effectively means that your only options are to have fully typed code or fully untyped code, because almost anything in between will run up super slow. Which kind of ruins the whole point of being able to gradually add types to your program.

> Thanks for the offer, I am definitely interested.

You can get very far by searching google scholar for "gradual typing. One good place to start out would be Siek and Taha's original paper on gradual typing for functional languages[1] and Sam Tobin-Hochstadt's PHD thesis on Typed Racket[2] (which used to be called Typed Scheme back then). For an example of the challenges of gradual typing in practice, this recent paper [3] might be interesting.

[1] https://cs.colorado.edu/~siek/pubs/pubs/2006/siek06:_gradual...

[2] http://www.ccis.northeastern.edu/racket/pubs/dissertation-to...

[3] http://www.ccs.neu.edu/racket/pubs/popl16-tfgnvf.pdf

> With this restriction, you cannot assign an accurate static type to a function with a dynamically typed implementation, which means that you can't use dynamically typed libraries from inside statically typed code.

It depends what you mean by an accurate type. A accurate type for adapting your example would be to expose Variant -> Variant as Int -> Maybe Int, thus capturing the partiality. I concede that Int -> Maybe Int cannot be used in a function that expects Int -> Int, but that is how it should be! An untyped language cannot provide me with a function of Int -> Int.

> It depends what you mean by an accurate type.

The gradual typing literature tends to prefer having partial functions (that might raise well-behaved type errors at runtime) because it eases the transition between typed and untyped programs. You can add or remove type annotations without having to simultaneously add and remove tons of explicit pattern matches against Maybe. But this isn't the part I was talking about.

The key issue is that when you cast a function from (Variant->Variant) to (Int->Maybe Int) you need to add a small wrapper around the dynamic function that converts the input from Int to Variant and converts the output from Variant to Maybe Int. In theory it is simple but in practice all those wrappers can add up really quickly...

Functions are the simplest case where these higher-order type-conversion issues pop up but in my experience things are actually the most thorny when it comes to mutable objects and arrays. The extra type checks and conversions around every property read and write can add a ton of overhead. And from a semantics point of view you need to be careful because if your implementation is based around wrappers it could end up messing with object identity (like Javascript's === operator or Python's "is")

GHC also considers Dynamic->Dynamic and Int->Int to be incompatible types, which is not quite what one would want in a gradualy typed setting.

Haskell also sidesteps the difficulties that come with mutability in a gradualyly typed setting.

Sort of. If you wrap a polymorphic type inside a monomorphic one then it can handle it fine, for example

    data Mono = Mono (forall a b. a -> b -> a)

An important aspect of type annotations, for example, seems to be that they help document the API of a module, regardless of whether they're statically checked. This is especially relevant with respect to maintenance tasks.

Conversely, the benefits of static type checking to get correct code initially seem to be a lot murkier. There seems to be a cost associated with static types and that is that it takes longer to write code; time that can be spent in a dynamically typed language on other validation work (remember that types do not even remotely capture all software defects).

This means that it is definitely worth exploring the benefits of optional/gradual/soft typing vs. static typing and/or runtime type checking vs. compile-time type checking.

Conversely, the assumption that static types are an unalloyed good is not currently borne out by the evidence (in part, of course, that is because the evidence is inconclusive).

But this is not the only cost that matters, indeed might not even be a cost.

I've gone from being neutral about static vs. dynamic types to being pro-static types -- and the change happened when maintenance became a bigger part of my job.

Writing new code is now far less important to me than looking at a small part of a large system and trying to understand what is and is a not possible at that point. Static typing does not make this easy, but dynamic typing makes it far more difficult.

I'm not saying otherwise. My point is that there's no objective way to proclaim one better than the other. This depends on application domain, economic constraints, engineering constraints, what you're doing, and so forth.

Writing Ada software that controls a pacemaker has totally different requirements than exploratory programming in Jupyter that mostly deals with integers, floats, and arrays and matrices thereof, for example.

Very true. But any analysis that emphasizes writing code over maintaining it will systematically bias itself in favor of dynamic typing.

Interestingly I have had the converse debate with some of my colleagues, who have learned to hate Python because they keep having to debug existing systems. I try to tell them that it is an excellent language for the kind of one-off data-analysis that I did when I was a scientist.

They don't believe me, because here among software engineers, seemingly innocent 400 line scripts keep growing into giant, decade old, 100kloc typeless monstrosities.

This is not what the studies do. There is no emphasis on anything. They look at how people do on a number of different tasks with different typing options and report the results.

Also, it's not just dynamic vs. static typing. Gradual and soft typing is also of interest, because it allows you to turn dynamically typed code into statically typed code without a complete rewrite.

The sweet spot for me at the moment is a language which is fundamentally static but has good type inference so that you don't actually have to mention types all the time.

Haskell's pretty good at this, although because it's possible to write many Haskell programs without actually mentioning a single type anywhere it can be a pain to read the results. Rust deliberately constrained type inference so that you have to have type annotations on functions, which makes sure your interfaces don't get too murky when reading the code.

I'll go with the idea in another reply that static typing really starts to show its benefit in long-lived code bases under maintenance.

Certainly a static type checker makes refactoring a lot easier, in my experience.

Note that I wrote statically checked. You can also check them at runtime (basically as a form of Design by Contract), which is what Dart does at the moment.

Runtime type checks allow you to greatly simplify the difficult parts of the type system or make it more expressive.

People often forget how many expressiveness shortcuts we take so that our type systems keep working out. We have integers (which are in actually practice often just elements of a finite ring), but introduce natural numbers and suddenly a lot of static type checks stop working out. How about interactions between the natural numbers with and without zero? Integers modulo your wordsize vs. infinite precision integers? Can you statically check overflow or underflow? Dart has both `num`, `int`, and `double` types, and `int` and `double` are subtypes of `num`, but `int` is not a subtype of `double`; readAsBytesSync() returns an `Uint8List` that conforms to `List<int>`. From a mathematician's perspective, computer science type systems are usually painfully simplistic. They usually capture something about the representation of the data, not its actual mathematical properties. Their role is often is to prevent undefined behavior, not to capture the actual semantics. Computer science typing often tends to be opportunistic rather than semantically oriented, based on what's easily possible rather than what one wants to express (see the entire family of covariance issues).

As an extreme example, consider GAP's [1] type system. We're dealing with computer algebra here, and some of the types simply cannot be captured at compile-time. Any type annotations would have to be checked at runtime, other than the trivial ones (and even integers are basically a subtype of the rationals or any polynomial ring with integer coefficients).

If I want to do effective implementations of matrices and vectors over finite fields, I pack several finite field elements into a machine word and extract them; no static type system in existence has even remotely the power to statically typecheck these operations for me, unless layered over a ton of unsafe code, and then only for some languages.

From the computer science side of things (I have a background in formal methods), type systems are underpowered specification languages that just pick a small subset of a formal spec that happens to be convenient to prove. They can express only a subset of program behavior (compare what types in even (say) Haskell do to a full-blown formal specification in Z) and often not even the interesting ones. As a result, I don't particularly stress out over whether checks only occur at compile time. Runtime type checking is simply a different tradeoff in that you move from the set of types that you can build a sound type system around to predicates that can be expressed at runtime. It's simply a different set of tradeoffs.

[1] http://www.gap-system.org/

Perhaps. As someone who has used Python and Haskell for roughly equal amounts of time in my professional career I definitely find Haskell faster to write in.

If you wanted to add variants to Dart, you'd either need to add this runtime information, or limit the Dart typesystem to be no more and no less than what the JavaScript typesystem already stores dynamically.

If Dart wants to distinguish ints and doubles, it needs to keep track of that on its own; if it has a JavaScript variable x and needs to know calculates x/2, it won't know whether to round it as if it were integer arithmetic, or return a floating-point value. Usually it is enough to do this at compile time (the function calculating x/2 knows whether it's halving a Dart-level double or a Dart-level int), and so there's no need to make x an object that remembers its own Dart type.

Also, even if JavaScript distinguished ints and doubles, a higher-level language is likely to want to have multiple kinds of things with an int representation: bools, enums, bitfields, Unicode scalar values, etc. Again, ideally that information is tracked at compile time and the knowledge of what to do with an int representation is inserted into the compiled code, and so no metadata needs to be stored at runtime.

(1) creating abstractions that are typed correctly is easy

(2) the existing abstractions you're building from are correctly typed

(3) the properties that the type system proves are valuable enough to reward the effort

Whether (1) is true depends on your type system. If it's very simple (e.g. dynamically typed), it's trivially true. If it's rich and expressive, it can sometimes be impossible to express your intent without obfuscating your types (e.g. using pointers and typecasts or polymorphism or some other loophole); and with some type systems, it's plain impossible to express some valid programs.

Whether (2) is true is a social problem. You need to have enough sensible people in your community such that you get to use List<T> rather than ObjectList with typecasts, or what have you. What's available in the runtime libraries often translates into what's commonly used for APIs in third party modules, and if the library of types isn't rich enough, APIs will use loopholes or typecasts or other mechanisms that obfuscate types. Third party code turns into a kind of firewall that stops your types (and the information, proofs, they convey) from flowing from one module to another in your program.

Whether (3) is true also depends on your type system, but also on your application domain. It's in direct opposition to (1); the more you can prove with your type system, the harder it normally is to write code that proves exactly the right things, and neither more nor less (the risk is usually too little type information flowing through). If the properties being proved are important to your application domain, then that gives a big boost to type systems that can prove those properties. If it's just because it gives you warm fuzzies, and testing is what proves interesting things because interesting properties (in your domain) are harder to prove with a type system, then rich type systems are less useful.

Personally, I was once a strong proponent of rich expressive type systems. But I eventually learned that they can become a kind of intellectual trap; programmers can spend too much time writing cleverly typed confections and less time actually solving problems. In extremis, layers of abstraction are created to massage the type system, often in such an indirect way that the value of the properties being proven are lost owing to the extra obfuscation. Now, I see much more value in a strong type system for languages like Rust, but much less value at higher levels of abstraction. That's because many of the properties being proven by Rust (like memory safety) are already proven in high-level dynamic languages through GC and lack of pointers and unsafe typecasts. Whereas application-level properties being proven through type systems are seldom valuable.

Documentation and tooling are much stronger arguments for types in high-level languages than proofs; ironically, this means that the type systems don't need to be sound, as long as they're usually OK in practice!