I agree that Scala works well in some skilled hands (see Li's libs), but the people that write the most terrible Scala programs actually know a lot about Scala. Scala has a unique way of encouraging genius programmers to write crazy code. People that know little Scala can just use it as a "better Python" language and actually write amazing Scala code.

I agree that Scala codebases that follow the Principle of Least Power can be amazing: https://www.lihaoyi.com/post/StrategicScalaStylePrincipleofL...

Scala programmers generally frown on programmers that want to use it as a better Python and I think it should be the opposite.

It's hard to make generalizations like this that are meaningful. There are a number of different Scala ecosystems, as you probably know. Spark is really a world of its own. Lightbend's libraries form their own ecosystem, as do Typelevel's and ZIO's.

My own opinion is that using algebraic data types (sealed traits, final case classes) with a sprinkling of type classes (implicits) where needed is pretty easy to work with, and that's the style that Typelevel is using. It is mostly inline with Li's "Least Power" post, though using IO pervasively instead of Future.

Is Typelevel's approach a better Python? I don't know. I don't really know what that means, and I'm not convinced that Python is a great model to follow. I see you posting along similar lines in many Scala threads on HN, so you seem quite invested in the language. I think it would help make your point clearer if wrote up a bit more of your thoughts, perhaps as a blog post.

The only Typelevel library I've used is frameless, which I blogged about here: https://mungingdata.com/apache-spark/frameless-typed-dataset.... Don't think the lib is really usable. Typelevel stuff is not what I'm referring to when I say better Python. I'll try to make another blog to clarify.

I think Scala is a better language than Python. But for some reason it just seems like Scala is full of mind bending code that I can't understand. Full disclosure: I am way less smart than lots of the Scala ppl I've worked with.

I find what most people mean when they say "simple" is really "familiar". When I first started teaching most programmers weren't familiar with first class functions and they were "complex". Now almost everyone has been exposed to them. Similarly I'm seeing more students are learning FP at university and thus find FP techniques "simple".

Scala is a popular platform to explore FP concepts without fully understanding their caveats or having enough respect for the rest of the team's ability and willingness to follow, I'll give you that. But that's only because it's a great platform to explore FP concepts in general!

You have to reign in the "geniuses", that's what engineering culture is for. I really don't like that label for people writing crazy code, btw. They shouldn't be on a pedestal just because they lack the empathy to write maintainable code.

The pythonic-Scala approach is certainly gaining steam in the ecosystem. The problem is, it's hard to see because it's boring and doesn't require many special tools – Scala has much of that built-in. So the popularity of this approach is not apparent if you just look at open source activity. But internally companies who work with Scala use it in very benign ways for the most part, in my experience at least.

Scala is almost as much joy to write data munging tasks as Python (unlike say C#, C++, Java, and I have to say Golang).

Chaining multiple maps and filters is so much more pleasurable than writing 4 nested loops with multiple ifs inside.

The better part is that you can reliably deploy Scala unlike Python.

We just had yet another discussion on the mess that Python deployment is: https://news.ycombinator.com/item?id=29439149

Yes, but, in my experience, the productivity gains are not that big. Strong developpers can and _will_ fare well in any technical environments, therefore choosing Scala over anything else won't give your team any advantage over a "simpler" technology. It's actually riskier to pick Scala, because you will increase your recruitment difficulty by a factor of 100-1000 (less developers available in the pool, even less when you only consider skilled developers, and huge barriers to teach and bring up to speed anyone that haven't worked in Scala before). What do you think will happen when the "good" developers are gone ? A rewrite in Python.

Save yourself, don't use Scala, pick Rust.

That's an hilariously bad advice if you think hiring competent developers and onboarding juniors is hard in Scala.

Most things companies do on the JVM, especially at scale, are going to be a lot harder in Rust and its ecosystem.

Absolutely not! I'm serious, and I stand by it. Rust ecosystem is like a better Python. You have to consider why someone is willing to move away from Python and into a foreign language: Developer supply, performance, safety, ecosystem, political reasons.

I can't cover all cases so I'm going to focus on the one that makes the most sense _from my personal point of view_.

I do AI Software Enginering and Devops when I'm not project managing. So Python is the only and primary choice for all things related to Data Science, dashboards, APIs, pipelines and transformations (on data).

Why and where would I choose Rust ?

In data pipelines, for performances and memory efficiency. This also ensure we have proper boundaries and documentation (APIs, file formats, schemas) between Data Science-related code and pipelines.

In Dashboards and APIs, for improving type safety, and handling more concurrency on background computations than python without resorting to other frameworks (Celery, Airflow, ...).

In data versioning and supporting infrastructure, for additional type safety and mostly making sure that _nobody_ who is not a software engineer won't cross component boundaries by accident. These components are often very delicate and essentials for a working platform. Changing them can have ripples into other parts of the project, and I only want trained engineers on these.

In the opposite, direction, I could pick Python over Rust when it's most appropriate: Restricted developper availability, improving coordination with Data Scientists, single container deployments, existing experience with framework or ecosystem...

The main blocker for not using Rust isn't because the language is new or more complicated, it's because Python offers more advantages to a situation. At the difference from Scala, I can pick _any_ skilled developper and put him from Python to Rust without any side effects (appart from the training part). This is because writing Rust code is much more consistent at all skill level than Scala. There no "flavor" of Rust. There no bickering about "coding style". The Rust tooling is best in class (Scala's SBT is actually worst-in-class, I believe). You can use the same IDE in Rust an Python (VSCode). This leads to a very similar coding experience between writing type-hinted-Python and Rust. This is not the case between Python and Scala.

While the criticism that Scala's tooling support might have been true some six or seven years ago, nowadays it is top notch.