Absolutely agree you can tweak Thompson sampling to handle nonstationarity, periodicity and delayed feedback. I even published the math for one variation of nonstationarity: https://www.chrisstucchio.com/blog/2013/time_varying_convers...

(I've also dealt with delayed reactions, but I've never published it, and probably won't publish until I launch it.)

Dynamic yield has the delayed feedback problem because users might see a variation in session 1 but convert in session 2 (days later). They "solve" this by doing session level tracking instead of visitor level tracking - the delayed feedback is now only 20 minutes (same session) instead of days.

The problem is that session A and session B are now correlated since they are the same visitor. IID is now broken.