The concept of references was introduced in my introduction to Java course I guess at the same time that the concept of objects were explained more fully. At least in Java, I feel that you have to understand the concept of references, to be mindful of aliasing, the fact that primitive values can't be aliased (I've tried twice to make a method to swap two primitive values, of course failing at that), and so on. Pointers don't really feel like a conceptual jump, coming from Java - the fundamental concept of indirection is still there. The fact that you can do pointer arithmetic and such on them, while error prone and harder to deal with than references, did not feel that hard to understand, conceptually. Granted, I have very little experience with C-level programming, so I be missing some subtle points.

There might be a difference for people who are used to languages which are more consistent than Java (like: "all values are really objects"). Do people not have to know about references in languages like for example Python?