You're missing the point. I agree that matching the predictions of experiments (not previous theories--I'm talking about experimental results that match the predictions of GR, not just those predictions themselves) is not a sufficient condition for a theory to be accepted (which is what you are saying); but it is certainly a necessary condition (which is what I was saying).
> Just recall how Kopernik's theory of solar system got accepted. It had worse predictions than Ptolemy's scheme at the time it was introduced
Yes, and it wasn't accepted at the time it was introduced. Actually, Copernicus' theory in its original form was never really "accepted"; what was accepted was Kepler's reformulation using elliptical orbits, based on Brahe's more accurate observations. Kepler's model was more accurate than Ptolemy's, and that was a key factor in its acceptance.
I agree with you that if a new theory was to replace the old one for making specific set of predictions, it should give predictions of similar or better accuracy. But I do not think that replacement is necessary for the new theory to compete or be accepted; it is the new benefit it brings, whatever its nature may be, that is crucial. The two can temporarily both be accepted to coexist, if both have their strengths. For example, quantum theory does not make the same predictions as classical theory when it comes to classical experiments (mechanics, basic EM phenomena) and is largely useless in that domain. It only gives probabilities of results of specified experiments of certain kinds; it does not reproduce the old predictions (like definite trajectories, Moon phases or solar eclipses), but provides new results (like resonance frequencies of atoms and molecules and their bond energies). Similar thing can happen with a new theory of gravity; it may not give the same prediction for Mercury perihelion precession, but it may be able to explain other things, like why the inverse square law, why no repulsive gravity or why the mutual gravity force between electrons is so much lower than the mutual EM force. Explanation for oddities in Mercury motion could then wait for further data and repetition of calculations. It is natural to expect of any new theory to bring new results, but demanding that it reproduces all the old ones along is too much. That happens rarely and such expectation only prevents any new ideas from being considered.
> quantum theory does not make the same predictions as classical theory when it comes to classical experiments (mechanics, basic EM phenomena)
Yes, it does. Do you know how the classical limit of quantum theory works? That limit is what allows us to use classical physics in the domain where it works. If that limit didn't work, we would have a serious problem with consistency.
> It only gives probabilities of results of specified experiments of certain kinds; it does not reproduce the old predictions (like definite trajectories, Moon phases or solar eclipses)
Are you aware that all of those "old predictions" can indeed be derived from quantum theory, using the classical limit I described above? The reason that works is that, in the classical limit, quantum theory predicts a probability of 1 for one result--the classical result.
> It is natural to expect of any new theory to bring new results, but demanding that it reproduces all the old ones along is too much.
You appear to have a mistaken understanding of how new theories get accepted. New theories that don't reproduce all of the predictions of the theory they replace, in the domains where the old theory is verified by experiment, are not accepted. If general relativity had not reproduced all of the predictions of Newtonian gravity in the weak field, slow motion limit, it would not have been accepted. And if quantum theory had not reproduced all of the predictions of classical physics in the classical limit, it would not have been accepted.
As I wrote above, I agree with you on the requirements for replacement theory. My point is that a new theory of a phenomenon does not need to replace and reproduce all the results of the old theory to be considered worthwile, competing, acceptable.
Can you give an example of a new theory that was considered worthwhile even though it didn't replace and reproduce all the results of the old theory? I'm not aware of any. (The Copernicus example given upthread is not a valid example, as I said in response to that post.)
Schroedinger's theory of hydrogen atom and his wave mechanics (1926). It explained positions of emission lines of excited hydrogen, but it didn't explain how the atoms lose excitation energy as there is no c and no spontaneous emission in that theory. Larmor's older theory (1897) explained how the energy is lost - by EM radiation - and gave formula connecting acceleration and losses that is used to this day.
Joseph Larmor, LXIII, On the theory of the Magnetic Influence on Spectra ;
and on the Radiation from moving Ions, Philosophical Magazine Series 5
Vol. 44, Iss. 271, 1897
Erwin Schrodinger, Quantisierung als Eigenwertproblem. Annalen der Phys. 384 (4) (1926)
> Schroedinger's theory of hydrogen atom and his wave mechanics (1926).
This was not a "new theory" that was competing with any "old theories". It was a tentative model in a regime where no previous theory existed, and it was never claimed to cover anything outside that limited regime. It wasn't competing with any other theories, because there were no other theories to compete with. The question of whether or not Schrodinger's model reproduced the predictions of the "old" theory never arose, because there was no "old" theory. (Technically, there was a sort of "old" theory of the hydrogen atom--Bohr's model--but Schrodinger's model did reproduce all of its correct predictions, plus it added more correct predictions of things that the Bohr model got wrong.)
The position with regard to gravitational waves is very different; we already have a comprehensive, fundamental theory--General Relativity--that explains them. Any alternative theory that only explained GWs, and didn't also explain all the other experimental results that GR explains, would be a nonstarter.
> Larmor's older theory (1897)
This wasn't a separate "theory" at all; it was just a derivation of a particular formula using an already known theory, Maxwell's Equations.
Schroedinger theory certainly was a new theory of the atom and later of molecules at that time, successfully competing and largely replacing classical EM models of atoms and molecules such as Larmor's theory of molecules, although it didn't cover the EM radiation aspect and EM theory needs to be used in parallel with Schroedinger's to get, say, intensities of emission lines. I think this is a good example of what I was saying in the first post. It is the new benefit that the theory brings, not reproduction of every single result of the previous theories, that makes the new theory interesting and helps its adoption. Cases where the new theory completely replaces the old theory and reproduces all of its positive results happen too, but are not the only way how new knowledge is adopted.
> I think this is a good example of what I was saying in the first post. It is the new benefit that the theory brings, not reproduction of every single result of the previous theories
Of course Schrodinger's model didn't reproduce the results of classical EM with regard to the atom. It wasn't supposed to, because those results of classical EM were wrong. In other words, there wasn't a correct "old theory" that covered the regime the Schrodinger model covered (the atom)--there was only a wrong "old theory".
As far as using Schrodinger's model plus classical EM theory to get results like emission line intensities, there also there was no correct "old theory"; there was only a wrong "old theory" (classical EM by itself, which did not predict emission lines at all, let alone their intensities--it predicted a continuous emission spectrum). Also, this hybrid classical-quantum model was known to be incomplete at the time; it was only used because nobody had yet figured out how to quantize the EM field.
> It is the new benefit that the theory brings, not reproduction of every single result of the previous theories
Once again, this is not the situation under discussion in this thread (gravitational waves). In the case you describe, the results of the previous theories were wrong in the regime the new model covered, so there was nothing to reproduce; there was no correct "old theory" for the new theory to compete with.
In the case of gravitational waves, we have a correct "old theory"--General Relativity--so any new theory that did not match that correct old theory would be a nonstarter. I am not aware of any case where a new theory was accepted as interesting when there was a correct old theory covering the same regime and the new theory did not reproduce its results.
> It wasn't supposed to, because those results of classical EM were wrong.
You're badly mistaken. Although nobody succeeded in obtaining the emission line frequencies of gases out of the classical EM theory, the theory did correctly give other results consistent with observations. One of them is the formula for emission intensity that connects energy radiated with second derivative of electric moment; it goes back to Larmor's work. This was the result the new theory would preferably reproduce or at least be consistent with. Wave mechanics wasn't consistent with it - the hydrogen atom oscillates indefinitely in wave mechanics. Schroedinger himself viewed this as a deficiency and planned to get back to it - check the ending part of his seminal papers on wave mechanics. The classical formula is taught to this day both in macroscopic EM theory and quantum optics courses, although there are some deficiencies and problems about the formula that Larmor did not know.
> In the case of gravitational waves, we have a correct "old theory"--General Relativity--so any new theory that did not match that correct old theory would be a nonstarter.
I do not think any physics theory could even be "correct" in the sense of Platonic ideals, but I do not know what you mean by "correct". I do not claim a new theory could completely replace the old one before it could deliver the same or better results. I claim theory has value and is accepted based on its new benefits, not its superiority in every aspect the old theory was superior before. Calling incomplete theory non-starter makes no sense to me, as all theories, including General Relativity, are incomplete.
No, I'm not; you're just mistaken about which classical results I was referring to. I meant the results of classical EM that predicted that atoms could not exist--because the electrons would radiate until they fell into the nucleus. And what classical formula tells you how much the electrons will radiate because of their acceleration due to responding to the electric field of the nucleus? Larmor's formula.
In other words, Larmor's formula was not a "theory"--it was a particular result derived within a theory. The particular result happened to be correct, within a particular limited domain; but the underlying theory that was used to derive it could not explain why it was correct--because the same theory, and indeed the same particular result--the same formula--made other predictions that were obviously egregiously wrong (like predicting that atoms would collapse).
> nobody succeeded in obtaining the emission line frequencies of gases out of the classical EM theory
You're drastically understating the failure of classical EM here. It's not that classical EM couldn't predict the particular frequencies of emission lines. It's that classical EM couldn't predict the existence of emission lines at all. Classical EM predicted that atoms would emit a continuous spectrum of radiation--not radiation sharply peaked at particular frequencies.
> The classical formula is taught to this day both in macroscopic EM theory and quantum optics courses
Sure, because within its domain of validity, it works fine as an approximation. But that's all it is--an approximation. And we explain why the approximation works, and why it works only within a particular domain of validity, by reference to the more complete underlying theory--quantum electrodynamics.
> I do not know what you mean by "correct".
I mean "makes predictions that match the results of experiments".
> all theories, including General Relativity, are incomplete.
I agree; but there's a big difference between:
- A theory that is incomplete because it doesn't cover absolutely everything, including where we haven't tested yet and won't be able to for the foreseeable future, but makes correct predictions everywhere we can actually test it; and
- A theory that is incomplete because it makes predictions about some things that are obviously at variance with observation, even though it makes correct predictions about others.
GR is an incomplete theory in the former sense; and theories that are incomplete in that sense can still be used to safely rule out competing theories that don't match their predictions in regimes where those predictions have been extensively confirmed.
However, classical electromagnetism is an incomplete theory in the latter sense; it made obviously wrong predictions, like the ultraviolet catastrophe and the instability of atoms. And even the correct predictions it made, like using the Larmor formula to predict radiative properties of atoms, were only obtained by using the theory inconsistently: by first assuming, contrary to the classical EM prediction, that atoms could be stable at all, and then working out what classical EM said about how these impossible objects (impossible according to classical EM) could radiate.
In a situation like that, you can't safely use the theory to rule out other theories, because the theory contradicts itself, and you can prove anything from contradictory assumptions. That's why classical EM physicists couldn't say "well, the Schrodinger theory can't be right, because I can't use it to derive the Larmor formula". You can't consistently use classical EM to derive the Larmor formula either; you have to sweep certain things under the rug and wave your hands that somehow or other it's ok.
In a situation like the latter, yes, you're right that anything that can give some handle on making predictions is going to be at least tried. But that's a very different situation from the former situation, where we have a correct theory that, within its domain of validity, doesn't have any of those issues. The only issue with GR is that it's not a quantum theory, which means, in the eyes of many physicists, that it's incomplete; but that incompleteness has no practical consequences whatsoever. It certainly is not a reason to entertain alternative theories of gravitational waves that get other predictions wrong that GR gets right.
> Just recall how Kopernik's theory of solar system got accepted. It had worse predictions than Ptolemy's scheme at the time it was introduced
Yes, and it wasn't accepted at the time it was introduced. Actually, Copernicus' theory in its original form was never really "accepted"; what was accepted was Kepler's reformulation using elliptical orbits, based on Brahe's more accurate observations. Kepler's model was more accurate than Ptolemy's, and that was a key factor in its acceptance.