> they can't be values because you can use them to modify other values
That seems like an arbitrary criterion. They're values because they behave like values when copying them around etc. You can't even use them to modify other values implicitly - you still need to use * to get an actual dereferenceable/assignable lvalue out of the pointer.
What operator is used to modify the value doesn't matter, just the fact that you can do so. At the point where you use an assignable lvalue, the optimizer needs to know where that lvalue came from, which means that pointers need to be tracked like references.
My point is that because pointers act like both values and references, they are neither values nor references. This makes it impossible to soundly reason about them.
It is certainly possible to soundly reason about them. The C standard just does not describe how to do it.
There is plenty of PL work (in particular everything considering ML with imperative features, many decades worth of work) where references are values, that works perfectly fine.
That seems like an arbitrary criterion. They're values because they behave like values when copying them around etc. You can't even use them to modify other values implicitly - you still need to use * to get an actual dereferenceable/assignable lvalue out of the pointer.