I was reading this answer on how Solomonoff's theory of inductive inference can be used to posit the more correct theory amongst a set that provide the expected "answer", where the shorter, or less complex, of the set has the highest likelyhood of being correct.
Are there any examples of theories where this has not held in the past?
Certainly. Here are a few:
General relativity replaced newtonian gravity as the more correct explanation, but it is substantially more complicated. Note that for small mass densities, they yield the same results, but for huge mass densities, newtonian gravity yields the wrong result.
Neutrino interactions via boson exchange replaced the earlier Fermi interaction model but are very much more complicated. Note that for low energies they yield the same result but the Fermi model fails at higher energies.
Special relativity replaced galilean invariance in electrodynamics as the correct formalism but is more complicated. Note that for low velocities they yield the same result but at high velocities, the galilean model fails.
The simpler models were perfectly useful and considered correct until circumstances were discovered in which they failed.
I was reading this answer on how Solomonoff's theory of inductive inference can be used to posit the more correct theory amongst a set that provide the expected "answer", where the shorter, or less complex, of the set has the highest likelyhood of being correct.
This is just wishful thinking. While a certain elegance and economy is a good feature for a theory to have (especially if you want others to use it), a theory will be as simple as required to predict behavior with a given degree of precision or accuracy but not simpler. In general, the higher the precision or accuracy, the more complex the calculation.
For an excellent example of two competing theories that described the same phenomena differently, Google Fenyman and Schwinger and QED. Feynman's approach was the simpler and is the one taught in textbooks. Schwinger's math was more elegant but few could follow his approach.
In physics (and probably elsewhere) we don’t propose models as if they describe what’s ‘really happening’ as distinct from ‘what appears to be happening’. Models, axis the name suggests, only claim to provide a way to predict what will be observed in particular situations. So any model that is consistently accurate is correct until it is shown to be incorrect. Occam’s razor tells us to go with the simpler model because it’s easier to work with but that doesn’t make the more complex model wrong, only finding an example where a model doesn’t work can prove it wrong.
Given that accuracy is the same I would argue that the method which is easiest to calculate or use would be considered "more correct" by humans. The Ptolemiac model of planets in celestial motion is said to be accurate enough for use by ancient navigators. Galileo is credited with causing a paradigm shift toward the heliocentric model of planetary motion by reference to Copernicus with compelling arguments that a body on earth, like a body in heaven, would remain in a persistent state of regular motion in the absence of resistance. This meant the earth could be in perpetual motion around the Sun in the same way that we regarded planets and stars in perpetual motion around the earth! Kepler's math for a body in orbit is much simpler than Ptolemaic calculations and both might give the same accuracy depending on the measurement methods and calculation errors.
Experts in numerical analysis and numerical methods also consider calculation error where the math model introduces error in the solution. Two models might be theoretical accurate but one model might introduce more calculation error when applied in an algorithm. Using a computer to solve the Quadratic equation for its roots can introduce significant error particularly if one develops the algorithm without reference to the literature on the problems inherent in the art of numerical methods.
