I've had this problem with Readability before (the icons appeared by default to the left of the back button, which I didn't like; they kept reappearing). I want to say I solved it by going to about:support and resetting things.