About 30 years ago, I had the very same experience as you. A very fast mouvement of the plane to avert a collision during an air traffic controllers strike over Paris. At the time, I was flying about twice a week. For about 18 months, I was so frightened during my flights that I was feeling exhausted when bording off the plane.
I finally cured it by learning (in principle) how to pilot a plane, by reading books and using MS Flight Simulator (on an IBM PC/AT at the time !). It completely cured me. Since then, I have flown hundreds, maybe thousands of times without any fear. I amuse myself by watching the knuckles of my neighbours when the plane moves a bit : if they turn white, it means that they are feeling very nervous about it! It's a lot more common that one thinks.
Good luck!