Environmental exposure is major factor, especially with heavy metals in the diet. Heavy metals can drive inflammatory responses and metals like lead are associated with cognitive decline. Lead exposure historically was associated with high-traffic locations (typically poorer people live closer to high-traffic zones, which also leads to asthma).
This could lead to all kinds of confounding factors, i.e. people with different lead exposure history could respond differently to 'anti-inflammatory diets'. Then there's liver damage; the liver processes many toxins and unprocessed toxins could cause neural damage and dementia, there's literature on that as well.
This is a self-report survey about historical diet. It's basically meaningless regardless. People that think they're healthy will report that they ate a lot of foods they currently think are healthy, not what they actually ate. This is true even if people are asked days later, much less years.
Wait, we're not adjusting for wealth, environmental exposure, exercise, fitness, or existing health conditions?