I don't particularly agree with the grandparent that getting rid of pointer-to-integer casts makes sense, but at least the specific question you're asking doesn't seem like a non-starter. Two options that come to mind immediately would be making a union { intptr_t; foo * } well-defined; or adding a 'p' or 'ptr' literal suffix so 0x80004000p has type void *.
Type punning in general is undefined (unless one of the types is char * or equivalent); but defining it between intptr_t and pointer types in a few restricted contexts (e.g. for const variables of static storage duration only) would be sufficient to address memory mapped registers and restrictive enough that it should be possible to define clearly.
