I’ve seen a lot of people taking issue with the self argument. Am I crazy to actually love it as a feature? I pulled out a lot of my hairs when I first started OOP (Java) trying to gork `this`, and I remember myself thinking “why didn’t everybody just do this” when I saw how Python declare methods. And I still think that’s a good idea now. So much easier to keep my sanity than using JavaScript’s bind.
I don't think you understand what is actually going on under the hood here, at all. Methods are member functions of a class, and must be invoked on an object.
someInstance.foo()
When invoking a method from within the class, you still need a reference to the object the method will be invoked on. In python, the reference is passed implicitly as the first argument, and usually called self. That has nothing to do with the method name or signature itself, and calling the method self.foo() would result in calling it like
someInstance.self.foo()
for cases when you aren't referencing the function from within the class, which would be even more confusing.
The syntax of a language is an abstraction over what is actually going on under the hood. There's no reason python couldn't have designed their OOP abstraction more elegantly.
> There's no reason python couldn't have designed their OOP abstraction more elegantly.
Sure there is: consistency. If member functions on an object that happens to be a class (i.e., methods) did magic transformations that member functions of other objects did not do, the mental overhead to understand Python code would be higher, and the ability to build custom general abstractions would be weaker.
It would perhaps make the simplest cases microscopically easier, but at the expense of making the already hard things more confusing and difficult to wrap with generalities.
Most statically-typed OOPLs don't have first-class classes that are just normal objects, so this isn't an issue because the things that enables aren't available; other dynamic languages may use models where methods aren't data members of classes (e.g., Ruby where methods are associated with classes, but not as instance members which, in Ruby, wouldn't be externally accessible—while Ruby classes are objects, the ability to hold methods is a special power of Ruby classes/modules compared to other objects, not just having instance members. This is one way Ruby's model is more complex than Python’s, and it definitely bites you in terms of seeking general solutions some times..)
Why is self.foo more elegant? It's very rare that you'll call a given method as self.foo, and there's a slew of complications that come with such a syntax (the declaration grammar is more complex, `self` is now special, etc.)
It's true that languages are abstractions, but not all abstractions are useful.
It doesn't have to treat "self" as a special keyword - it just has to desugar "foo.bar(x, y)" into "bar(foo, x, y)". There are some other languages that do that - e.g. in F#, you write:
member this.foo(x, y) = ...
Again, "this" is just an identifier here, and doesn't have any special meaning.
It's more elegant firstly because it follow use, and secondly because it means that "def foo(x)" has the same meaning both inside and outside of a class declaration - it's just a function, and there's nothing special about its first argument. As it is, we need stuff like @staticmethod and @classmethod. It's especially annoying when you have class attributes that happen to reference a function, because conversion from functions to methods is a runtime thing - so you have to remember to wrap those in staticmethod() as well.
> As it is, we need stuff like @staticmethod and @classmethod
You would need those even with your suggestion (except you could drop @staticmethod if you add another layer of magic so that methods declared without an leading identifier were assumes static; you'd still need @classmethod some equivalent mechanism to distinguish which non-static methods were class vs instance methods.)
My suggestion implied removing the descriptor-based "magic" on regular functions that make them behave as methods when accessed using dot-member syntax, and instead making the "def self.foo" syntax produce a special kind of function that would have that behavior. @staticmethod today basically just suppresses that special behavior on regular functions, so it wouldn't be needed in this case. But yeah, we'd still need @classmethod.
No it doesn't. Currently `def foo(self): pass` is called as `instance.foo()`. You're suggesting that `def self.foo(): pass` would be called as `instance.foo()`, except now it looks like self and instance are syntactically related in ways that they aren't.
> Again, "this" is just an identifier here, and doesn't have any special meaning.
But the grammar is no longer LL(1), and you have weird conditionally valid syntax, like `.` in a function name is valid only in a class block.
> "def foo(x)" has the same meaning both inside and outside of a class declaration
This is a stretch, especially since you're now optimizing for the uncommon case. Staticmethods are rare compared to instance methods (I'll go further and claim that static methods are an antipattern in python, modules are valid namespaces, you can stick a function in a module and couple it to a class also defined in that module and nothing bad will happen. Banning staticmethods entirely doesn't reduce expressiveness). Aligning staticmethods with functions, instead of aligning instance methods with functions (as python does currently) encourages you to do the wrong thing.
> classmethod
Your changes don't affect classmethod at all, if anything they'd make classmethod more of a special case. How do you signal that `self.foo()` takes `self` as the class instead of the instance?
> It's especially annoying when you have class attributes that happen to reference a function, because conversion from functions to methods is a runtime thing - so you have to remember to wrap those in staticmethod() as well.
What do you mean? Like
class Foo:
a = Foo.func()
@staticmethod
def func():
return 1
I'll say again: staticmethods are an antipattern in python:
def func()
return 1
class Foo:
a = func()
works just as well, better in fact. Modules are great namespaces. Classes are more than namespaces, and if all you need is a namespace, you shouldn't use a class.
> because conversion from functions to methods is a runtime thing
I'd also quibble with this: it's a binding thing.
class Foo:
a = Foo.foo(None)
def foo(self):
return 1
will work, and if you check, type(Foo.foo) is still just `function`, its only when you create an instance of Foo that the function `foo` is bound to the instance, and when that is done, the bound `foo` is converted to a method object. This was different in python2, where Foo.foo and instance.foo were both "instancemethod" objects, but in python3, Foo.foo is a plain old function, and instance.foo is a method.
Specifically this means that if you can get your hands on the `method` constructor (like with `type(instance.method)`), you can then do silly things like
and this will work. You'll have bound the function to the instance. Of course, if you stick an attribute on `instance` (or `A`), and reference `self.attribute` in the function, this will still work. (this also lets you do things like bind a given instance of a function to a different instance of the class, but that's because the method constructor is essentially just partial with some bookkeeping for class information)
Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).
But also, there's no reason to make those legal only inside classes. All it needs to do is make "def foo.bar" produce a different type of function, that has the method descriptor-producing behavior that is currently implemented directly on regular functions.
As far as less vs more common case - I think it's more important to optimize for obviousness and consistency. If "def foo" is a function, it should always be a function, and functions should behave the same in all contexts. They currently don't - given class C and its instance I, C.f is not the same object as I.f, and only one of those two is what "def" actually produced.
What I meant by function references inside classes is this:
This blows up with "TypeError: <lambda>() takes 0 positional arguments but 1 was given", because lambda is of type "function", and it gets the magic treatment when it's read as a member of the instance. So you have to do this:
Foo.bar = staticmethod(lambda: 123)
and even then this is only possible when you know that the value is going to end up as a class attribute. Sometimes, you do not - you pass a value to some public function somewhere, and it ends up stashed away as a class attribute internally. And it all works great, until you pass a value that just happened to be another function or lambda.
On the other hand, this only applies to objects of type "function", not all callables. So e.g. this is okay:
Foo.bar = functools.partial(lambda x: x, 123)
because what partial() returns is not a function. Conversely, this means that you can't use partial() to define methods, which can be downright annoying at times. Suppose you have:
class Foo:
def frob(self, x, y): ...
and you want to define some helper methods for preset combinations of x and y. You'd think this would work:
except it doesn't - while frob_xyzzy() and frob_whammo() both have the explicit "self" argument, they aren't "proper" functions, and thus that argument doesn't get treated as the implicit receiver:
Which is to say, this all is a mess of special cases. You can argue that this all isn't really observable in the "common case" - the problem is that, as software grows more complex, the uncommon cases become common enough that you have to deal them regularly, and then those inconsistencies add even more complexity into the mix that you have to deal with - just when you already thought you had your plate full.
> Python grammar is not LL(1) in general; just look at set and dict literals. This is really no different than ":" after an expression being legal inside a dict literal (and how you know that it is a dict literal).
Yes it is[0]. LL(1) Grammars can still be recursive, they just can't change the parsing rules based on distant context.
> As far as less vs more common case - I think it's more important to optimize for obviousness and consistency
Yes, but having the "easiest" thing you do:
class Foo:
def bar():
pass
silently do a usually unwanted thing (create a staticmethod) instead of an obviously wrong thing (raise an error) isn't obvious. It's building a footgun into the language.
The rest of your comment complains about inconsistencies of how python converts various callables to methods. This is a fairly valid and interesting complaint, but has nothing to do with syntax, it is solely a semantic complaint that would be solved by having class creation treat all attributes that are callables as functions. In fact, you could customize class creation yourself this way using __new__, no syntactic changes required.
> as software grows more complex, the uncommon cases become common enough that you have to deal them regularly
While this is true, I think you vastly overestimate how common these constructs are. Like, you're in the realm of "this doesn't appear on github" levels of uncommon.
Personally, again, I think "you can't use partial() to define methods" is a very good thing: if you're doing this, you're into weird metaprogramming land. Its not any harder to, for example, write out
Why is it an unwanted thing? By the same logic that demands "self" to be explicit, if you don't specify "self", then you explicitly don't want the method to be an instance method - seems very straightforward to me. And it's still invokable via instance member access, so the user of the class is none the wiser. Where's the footgun?
My complaint is of course more complicated than the syntax alone. I'm just saying that a distinctive syntax for self-as-receiver could be used to drive other changes that would make things more intuitive and self-consistent overall. And, of course, any discussion about "elegance" is going to be inherently subjective.
With respect to commonality of various constructs - this is all from personal experience writing Python code (for a project that is hosted on GitHub, by the way). It doesn't require particularly fancy code to trip that wire in general - it just requires code that tries to be generic, i.e. not make more assumptions that it needs to about the types of values that flow through it. Python classes break that genericity by treating functions, and only functions, in a special way whenever they flow through class attributes. This is particularly egregious in a language where every object is potentially callable, and non-function callables are very common; so functions really aren't all that special in general - except for that one case.
With partial() specifically, of course you can avoid that in this manner. But why would you, if it works with regular functions? I prefer it over defs, not just because it's more concise and avoids repeating things, but because it's also clearer - when you see partial(), it immediately tells you that it's a simple alias, nothing else.
But regardless, it's a function that has a certain documented behavior, and common sense would dictate that this behavior works the same for methods as well as functions. That it doesn't is not an intentional limitation of partial() - it's an unfortunate quirk of the design of methods themselves. And if you don't know exactly how functions become methods in Python, you wouldn't have any reason to expect the behavior that it exhibits. That's why the docs for partial() have to spell it out explicitly: "partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up" - because that's not a reasonable default assumption!
The bigger problem is that every library that offers a generic wrapper callable has to add the same clause to its docs, because they're all affected in the same way. And if they don't document it, and you use, say, a decorator from a library - how do you know whether the fact that it returns a function and not some other callable is part of the contract that you can rely on, or an implementation detail? Conversely, whenever you implement a decorator, you have to be cognizant that changing the type of the return value from/to plain function can be a breaking change for your clients - and that is even less obvious.
Let's take the following code snippet as an example:
class Foo:
def bar(val):
foo = val
I claim that most of the time this is a mistake, and the author would have preferred `def bar(self, val): self.foo = val`. In current python, this will raise an exception when called. In your proposed python, this will silently do nothing, possibly leaving the instance in an invalid state. This is a footgun. I admit the example is contrived, but forgetting `self` is a thing I've seen happen, and having it fail loudly is preferable to having it do something likely unintended. Again, if someone wants to do the unusual thing, `@staticmethod` is still around.
> With partial() specifically, of course you can avoid that in this manner. But why would you, if it works with regular functions?
Simply: because I'd prefer it if functions look like functions. Understanding that `def x` is a callable is easier than trying to discern if `x = foo(other_thing)` results in x being callable or not, where it does for some values of `foo`, but not for others. Which isn't to say that python shouldn't make this change, I think I mostly agree with your complaint, I just probably wouldn't take advantage of it.
> My complaint is of course more complicated than the syntax alone.
To be frank, I don't see any connection between your syntactical suggestions and your semantic ones. They seem to be entirely orthogonal.
