My understanding is there was no word in English for the word orange before the fruit was widely introduced in Europe. In other words the fruit gave the name to the color, not vice versa. (Other orange colored things were usually called 'ginger' or 'red-yellow'.)
The history of color words is a really interesting topic. For example, in Turkish the word for "brown" is kahverengi, which literally translates as "coffee-colored". Also, turquoise comes from the French word for Turkish, used to describe the color commonly used on Ottoman glassware.
