1. The point of scientific theories is not to make predictions; it is to put together stories that explain data. Models with explanatory power usually make predictions. When they do not, it is for one of three reasons: (a) the explanatory power was illusory; (b) all the experiments were done a long time ago and there are several rules-of-thumb describing them that the model finally puts together, but it makes no new predictions; (c) experiments that would test the model are infeasible. (a) is what is usually meant by a theory lacking predictive power but really this almost always means that the theory needed an inordinate number of fit parameters to retrodict anything, i.e., there were a very large number of other theories with the same level of simplicity that would have been equally possible. (This is the case with religious accounts of anything.) As for (b) and (c) they are irrelevant to the goodness of the theory -- though (c) might be a symptom of people trying to shield their theories from experiment, it needn't be. If for political reasons it became impossible to build any further accelerators, that would not reflect on the theories that might have been tested in these putative accelerators.
There are some results approaching the limit of type (b) scattered throughout physics, though I can't think of any pure cases. In general, understanding what the shapes of various graphs have to do with one another tells you something more, at the very least it tells you what features of a given material make it behave a particular way, and suggests what other materials you should be looking at. Nevertheless, in my experience it is not true that models are considered less worthwhile as they approach the limit of type (b) -- though for sociological reasons a model of this kind is less likely to stimulate further activity. (A case in point is Wilson's theory of phase transitions, which was understood to have revolutionized physics although, as far as I know, it made no predictions that were verified before Wilson got his Nobel Prize.)
2. Apart from theories of type (b), one is struck by the differences in attitude between scientists and people who really are interested in making predictions. Andrew Gelman had a sociologically interesting post up in November, arguing that it was sensible for Silver to pour reams of probably trivial factors into his election-forecasting model on the assumption that they might help. (Matt Pasienski had made some very similar points in an IM conversation.) From a scientist's perspective what Silver does is a fairly absurd case of "overfitting" -- one always learns to avoid unnecessary fudge factors but Silver just sort of heaps them on -- but of course his "model" is meant to forecast elections and not to explain them. If elections could be explained this would all be rather silly but the existing models are less than perfect, so arguably it makes sense to hedge one's bets.
3. Which brings us to the question of why overfitting is a bad idea -- and I don't mean egregious overfitting like having as many parameters as data points, but just the vaguely disreputable tendency to introduce random vaguely relevant factors to make your model fit better. I can think of three basic reasons: (i) models with many parameters are hard to use and don't correspond to the kind of simple mental picture that is usually necessary for new creative work, (ii) they leave more stuff unexplained (why are the parameters what they are?), (iii) they are, like the epicycles, increasingly difficult to refute. [One might also mention (iv) they violate Occam's razor, but I don't think it applies here as "necessity" is ill-defined as after all one's curves do get a little closer to one's data.]
4. Which brings us to Popper as interpreted by Godfrey-Smith. I think the best way to understand the rule against over-fitting -- and the related preference for simplicity -- is in these terms:
In this section I will use a distinction between synchronic and diachronic perspectives on evidence. A synchronic theory would describe relations of support within a belief system at a time. A diachronic theory would describe changes over time. It seems reasonable to want to have both kinds of theory. [...] epistemology in the 20th century tended to suppose we could have both kinds of theory, but often with primacy given to the synchronic side. The more novel possibility, which I will discuss in this section, is the primacy of the diachronic side, once we leave the deductive domain. [...] A diachronic view of this kind would describe rational or justified change, or movement, in belief systems. [...]
In this section I suppose that we do not, at present, have the right framework for developing such a view. But we can trace a tradition of sketches, inklings, and glimpses of such a view in a minority tradition within late 19th and 20th century epistemology. The main figures I have in mind here are Peirce (1878), Reichenbach (1938), and Popper. This feature of Popper's view is visible especially in a context where he gets into apparent trouble. This is the question of the epistemic status of well-tested scientific theories that have survived many attempts to refute them. Philosophers usually want to say, in these cases, that the theory has not been proven, but it has been shown to have some other desirable epistemic property. The theory has been confirmed; it is well-supported; we would be justified in having a reasonably high degree of confidence in its truth.
In situations like this, Popper always seemed to be saying something inadequate. For Popper, we cannot regard the theory as confirmed or justified. It has survived testing to date, but it remains provisional. The right thing to do is test it further. So when Popper is asked a question about the present snapshot, about where we are now, he answers in terms of how we got to our present location and how we should move on from there in the future. The only thing Popper will say about the snapshot is that our present theoretical conjectures are not inconsistent with some accepted piece of data. That is saying something, but it is very weak. So in Popper we have a weak synchronic constraint, and a richer and more specific theory of movements. What we can say about our current conjecture is that it is embedded in a good process.
Occamism has been very hard to justify on epistemological grounds. Why should we think that the a simpler theory is more likely to be true? Once again there can be an appeal to pragmatic considerations, but again they seem very unhelpful with the epistemological questions.
From a diachronic point of view, simplicity preferences take on a quite different role. Simplicity does not give us reason to believe a theory is true, but a simplicity preference is part of a good rule of motion. Our rule is to start simple and expect to get pushed elsewhere. Suppose instead we began with a more complex theory. It is no less likely to be true than the simple one, but the process of being pushed from old to new views by incoming data is less straightforward. Simple theories are good places from which to initiate the dynamic process that is characteristic of theory development in science.
5. Which, finally, brings us to Gowers --
I would like to advance a rather cheeky thesis: that modern mathematicians are formalists, even if they profess otherwise, and that it is good that they are. [...] When mathematicians discuss unsolved problems, what they are doing is not so much trying to uncover the truth as trying to find proofs. Suppose somebody suggests an approach to an unsolved problem that involves proving an intermediate lemma. It is common to hear assessments such as, "Well, your lemma certainly looks true, but it is very similar to the following unsolved problem that is known to be hard," or, "What makes you think that the lemma isn't more or less equivalent to the whole problem?" The probable truth and apparent relevance of the lemma are basic minimal requirements, but what matters more is whether it forms part of a realistic-looking research strategy, and what that means is that one should be able to imagine, however dimly, an argument that involves it.
This resonates with me because I've always had a strong formalist streak; it goes with the Godfrey-Smith quote because formalism in mathematics is a diachronic perspective -- it says, "mathematics is a set of rules for replacing certain strings of symbols with others" -- and I think a diachronic philosophy of physics would have some appealing resemblances to formalism. I was going to explain how I think a diachronic perspective and the effective field theory program might affect how one thinks about, say, the many-worlds interpretation of quantum mechanics, but this post is already far too long.