John,
I understand what you are saying, but on the other hand professional physicists are continuously deluged by the half-baked ideas of the public, and they understandably become inured to this kind of thing. Yet, my hope is similar to Lee Smolin's, which is that one of us non-professionals could help find the mountain in the landscape of possibilities that the professionals would then be able to scale. Did you happen to catch the Science Friday show with Brian Green and Lee Smolin? If you read my preliminary essay, you will find part of my original narrative on the show in there, but I'll repeat it here in its entirety, for the first time, just for you:
Begin narrative:
After discussing Smolin et al's criticism of string theory's failure to predict new physical phenomena, and the effective suppression of new ideas due to its thorough domination of academia, they turned to discussing the physics crisis itself.
As Green explained how tests of the consistency of string theory calculations, and comparisons with the established concepts of physics, show that the "theory comes through with flying colors every step of the way and keeps us thinking that things are at least headed in the right direction," Flatow turned to Smolin and asked, "Well, Lee, what would be wrong with that, if things are working like that?"
Smolin's answer was very telling, and it's well known in the community: In spite of these favorable things that one can say about string theory, there are some very important things that "it doesn't come close to doing," asserted Smolin. Then he hit the nail squarely on the head:
"If you really put quantum mechanics together with the description of space, then we know, from general considerations, that the notion of space should disappear. Just like the notion of the trajectory of a particle disappears in quantum mechanics, ...the same thing should happen to space and the geometry of space."
"So far, string theory doesn't address this very directly," Smolin said, "while other approaches do," referring, of course, to his own continuum-based theory of quantum gravity, called loop quantum gravity (LQG), but before he could explain this further, Flatow took a call from a listener who suggested that "thinking outside the box," is what's required, which momentarily distracted the conversation away from the idea that "the notion of space should disappear," in the union of quantum mechanics and the description of space (the spacetime of relativity theory).
Smolin replied to the listener's comment, stating that, while he agrees with her, he definitely feels that it has to be the trained minds of professional physicists that do the "out of the box" thinking, who "go back in the decision tree," looking for new answers to foundational questions, because only they are prepared to readily scale the true mountain of knowledge, once the location of the highest peak in the landscape is discovered. Whereupon Flatow interjected with the obvious conclusion, in the form of a penetrating question:
"Are we at a point now, where you just have to sit and scratch your head and think, "We need some revolution, don't we?" I mean, we need a revolution in physics; maybe, we need a new physics!"
However, Smolin's reply to this conclusion reveals just how difficult it is for the minds of professionals, trained from the start in what Thomas Kuhn termed "normal science," to think "outside of the box." Instead of agreeing with Flatow's conclusion that a "new physics" is required, he demurred. "Nothing can happen without experiments," he asserted laconically, deftly inferring a different meaning of the phrase "new physics," which is a phrase that today is commonly used to refer to new experimental anomalies, not a new foundation for theoretical physics, something that is nearly inconceivable to the professional physicist. Yet, the truth is, the trouble with physics is not due to a lack of available, inexplicable, empirical data, but to the fact that there is no satisfactory explanation of the existing data from many, many experiments, including the anomalous results behind the so-called dark energy and dark matter enigmas, and the famous Yang-Mills mass-gap problem, to name just a few.
Clearly, however, Smolin revealed his hand with his comment: While he's certainly dismayed with the emphasis on string theory research, which seeks to unify the discrete with the continuous through modification of the current discrete paradigm, with which the string theorists are most familiar, Smolin and company prefer to approach the problem from within the context of the current continuous paradigm, with which they are most familiar.
Smolin's argument is not that a new foundation for theoretical physics is required, but that a shift in academic research emphasis is required, from the "let's modify the existing discrete theory" to solve the problem (string theory), to the "let's modify the existing continuous theory" to solve the problem (loop quantum gravity). The pressing need, from Smolin's point of view, is to complete the "unfinished revolution," which Planck and Einstein started, by exploiting Einstein's concept of the continuum, in order to unite the disparate theories, instead of modifying Einstein's concept of the quantum, in order to unite them.
Yet, the most logical conclusion that naturally occurs to the non-professional, didn't escape Flatow: "We need a revolution in physics; maybe, we need a new physics!" he had interjected, implying the need for a completely new foundation for theoretical physics, which doesn't require the reconciliation of two, incompatible, theoretical concepts of space and time, one static and fixed, the other dynamic and changing, but finds a new concept of space and time that works as nature works; that is, a new concept that works as the dual properties of one component, where the discrete and continuous realities are simply two aspects of the same thing.
Truly, as unpalatable, as unlikely, and as inconceivable, as the prospect is to the many of the minds of today's practitioners of "normal science," the possibility that a totally new solution exists that would revolutionize existing discrete and continuous concepts, and that would explain the dual quantum and continuum nature of reality, as two aspects of the same entity, has to be regarded as a legitimate alternative that needs to be seriously considered, though it may seem iconoclastic to today's scientists.
Evidently, as the NPR interview continued, since Smolin had dismissed the possibility of a "new [theoretical] physics," which he had suggested, Flatow turned to Green to get his comments, and Green seemed more willing to admit that something truly revolutionary in physical concepts is needed in our conception of space and time:
"I full well believe that we will, when we do complete this revolution that Lee is referring to, have a completely different view of the universe. I totally agree with Lee, that everything we know points to space and time not even being fundamental entities...We think that space and time...rely upon more fundamental ideas...What those fundamental entities are...that make up space and time, we don't know yet, ...but, when we get there, I think that we will learn that space and time are not what we thought they are. They are going to morph into something completely unfamiliar, and we'll find that, in certain circumstances, space and time appear in the way we humans interpret those concepts, but fundamentally the universe is not built out of these familiar notions of space and time that we experience."
Flatow stumbled a little, trying to get his head around an idea of what this might mean in terms of changes to existing concepts, which prompted Green to add:
"It would change the very notion of reality...We all think about reality existing in a region of space and taking place through some duration of time, but we've learned that those basic ideas of the arena of space and the duration of time are not concepts that even apply, in certain realms, and if the notions of space and time evaporate, then our whole conception of reality, the whole container of reality will have evaporated, and we'll have to learn to think about physics and the universe completely differently."
End of narrative
Of course, I believe that this new notion of reality is that everything consists of nothing but motion, a reciprocal relationship of space and time. However, it took me two months of full-time effort to simply write a ten-page essay in 5000 words on it, because there is so much to say that it's easy to get off track, when trying to capture it succinctly.
Many of the things you mention are interesting, and I would love to discuss them with you sometime, but the challenge I face right now is to try to convince Green, Smolin and company, to take a serious look at this mountain that I'm writing about. I think it's worth scaling, but he's right - you and I are not prepared to do it. We can only try to get their attention. Sometimes I have wished that Steven Weinberg were my uncle. LOL!!