This is largely to taste. An apostrophe doesn't always mean it's possessive, as just demonstrated, but it is also used for plural abbreviations or for numbers (like 1960's).
Casual English is fairly fluid. People rarely talk exactly according to the rules of English class.
See, e.g., https://www.thepunctuationguide.com/apostrophe.html