Medical progress often starts with observations, and is confirmed by trials. Penicillin is one such example.
To discount the value of observations is to ignore an important part of the discovery process.
There is extremely limited data available around the use of nootropics for mind enhancement (as opposed to in medical use for treating illness). In the absence of of such data a survey like this indicates that there are a few things worth studying in more depth (and some supposed nootropics that seem worthless).
The author appears extremely aware of the problems with their methodology, and makes no attempts to hide these problems. To pull quotes:
The online nootropics community consists of mostly healthy individuals looking to use these substances to gain motivation or clearer thinking. They tend to “outpace” the scientific literature, using some substances that haven’t yet been shown to work, or haven’t been shown to benefit healthy individuals.
...
" I would have liked to have some placebo nootropics to use as a comparison, but since I wasn’t running a trial, I didn’t have that option. I did, however, try to assess the placebo effect in a couple of sneaky ways."
...
"I have very low confidence that these represent real effects rather than artifacts of the gymnastics I had to do to make the data correlatable at all, and I report them only to encourage other people to do more sober analyses."
I think attempting to do this - knowing that people will criticize the methodology - is admirable.
Observations are one thing, attempts to do any kind of serious study on the results are another. It is not meaningful to do statistics on data that bad.
No at all - you just need to be careful what conclusions we draw.
We can safely report the statistic that "more people self report caffeine having an impact than ginseng."
The author goes as far as noting how many people are reporting an impact where there appears to be no known mechanism that impact could be occurring. That's a defensible statistic too.
Medical progress often starts with observations, and is confirmed by trials. Penicillin is one such example.
To discount the value of observations is to ignore an important part of the discovery process.
There is extremely limited data available around the use of nootropics for mind enhancement (as opposed to in medical use for treating illness). In the absence of of such data a survey like this indicates that there are a few things worth studying in more depth (and some supposed nootropics that seem worthless).
The author appears extremely aware of the problems with their methodology, and makes no attempts to hide these problems. To pull quotes:
The online nootropics community consists of mostly healthy individuals looking to use these substances to gain motivation or clearer thinking. They tend to “outpace” the scientific literature, using some substances that haven’t yet been shown to work, or haven’t been shown to benefit healthy individuals. ... " I would have liked to have some placebo nootropics to use as a comparison, but since I wasn’t running a trial, I didn’t have that option. I did, however, try to assess the placebo effect in a couple of sneaky ways." ... "I have very low confidence that these represent real effects rather than artifacts of the gymnastics I had to do to make the data correlatable at all, and I report them only to encourage other people to do more sober analyses."
I think attempting to do this - knowing that people will criticize the methodology - is admirable.