≠I’ve been engaged in a long term conversation with a particularly stupid academic who can’t understand why we sceptics can be so certain we are right – when after all he is the academic who knows everything.
So, I was just musing how in theory one would explain to that type of academic why sceptics are so superior and it seemed to boil down to this simple statement:
“academics don’t understand natural variation”.
Given previous research into the nature of academia and sceptics, I would now suggest a very simple theory, which might be called: The Theory of Everything
Theory + Natural Variation = Everything
Or to use slightly different terminology: The Theory of the real world
Theory + Natural Variation = Real World
Or to put it other ways:
Natural variation is the deviation of the real world from theory
Academic Domain + Natural Variation = Engineering/Sceptic Domain
The reason for this discrepancy between the academic domain and that of the engineer/sceptic is that academics focus on theory. And for obvious reason theory only works in systems that are “well behaved”. That is to say a system where:
Theory = Real World + E(x)
Where there is exists a method such that for any arbitrary size of error (ε) such that E(x)< ε. The simplest such systems are ones where only white noise exists. That is to say, where any variation of any reading only affects that reading and doesn’t have “memory” so as to affect any other samples. In such a case, the more samples taken, the smaller the error so that E(n) = ε0/ √n. Therefore for any arbitrary error ε there exists a finite number n which will reduce the error function so that theory is close enough to the real world that for all purposes they can be considered the same thing.
However, like everything academia does, that’s fine in theory, but in reality the real world doesn’t work like that. Because in the real world, there are real budgets and real time pressures and real customers jumping up and down and real people twiddling all the wrong knobs. In the real world, things age, equipment drifts, new factors come into play and generally, the final value changes from the initial value.
So, in almost all the real world situations we have, there just is no practical method by which E(x) can be reduced so that it is practical to say “theory = practice”. And this is why engineering is both the knowledge and use of Theory, AS WELL AS all the many different factors which I’ve described as “natural variation” that cause theory and the real world to diverge.
1/f Noise (or in general 1/fn Where 0<n<2)
However, there is a particular class of problem, where not only for sound practical reasons doesn’t theory match reality, but even in theory – the theory will never match real world. And these are the class of problems which have “1/f noise”.
And the reason for this, is that when a system has 1/f noise, instead of:
E(n) → 0 as n → ∞
Instead
E(n) → √n* as n → ∞
*This is for the special case of 1/fn where n=1
Note! Rather than getting smaller, the error function is actually increasing with the number of samples. This will sound counter intuitive to anyone who has only been taught that “averaging gives a better indication of the value”, but to use an example, if we were to plot the position of a river … if we measure it once a day for a week, the fact we are repeating the same measurement does not make it better, however, because over time the river bank erodes, the more measurements we take, the further the last one will be from the first. Thus rather than the error getting smaller, the error gets larger the larger the number of samples.
Similarly, if we measure the weight of a person – it is very unlikely that we will be any more accurate with two measurements on successive days … because the weight of the person changes daily, even hourly. And over time they may put on weight or lose weight. So, again, in real life, things tend to diverge from their starting condition. And indeed it has been argued, that it is very much the exception that E(n) → 0 as n → ∞.
This is what the academic expects to get from “well behaved systems” where noise tends to average out:
As the length of sample increases, the variation from the “proper” value decreases such that longer and longer samples tend to become more and more like each other. But this only occurs in systems where there is no “memory” between samples and any variation only affects one sample. That way, because all noise perturbations are unique their sum is zero, so that each reduces in scale as root n, so that the total average error reduces as root n.
However, in systems where noise and variance affect more than one sample, this simple relationship does not hold. Indeed, if variance tends to sustain for longer than our entire sample period, then the variance increases the longer we sample. So, like the river system, we tend to get a values that diverge from the initial value like this:
In practice what this means is that if we take two groups of samples and compare the average, the shorter the sample length the closer they match. This is the kind of behaviour we see for the position of an actively eroding river bank, it is what we get if we measure body weight, if we measure temperature in most natural systems (where there is either active cooling or warming). It is what we get with instrumentation (which goes out of calibration).
But, in practice, all systems tend to have some of each type of variation (as well as other types like cyclic variations). Some of the variance affects only one sample (the noise frequency is much higher than the sample frequency). But some of the noise affects some and some all the samples. To take another simple example, imagine the random walk of a drunk. As they move, their hands and legs move, so initially, there is some degree of certainty as to where “they are”. So, e.g. if we fitted a single location point on their belt – even if they were “stationary”, they would be swaying …. and taking more than one sample would mean the average had less variance than any sample. But as they start to walk, their whole position changes. Now if we assume a random walk, their average constantly moves – and now the variation in the average is much greater than the variation in any sample.
Why doesn’t climate theory match the real world
This is a problem that perplexes academics involved in climate, because they come from an academic culture where theory is supposed to match the real world. In contrast sceptics come from real world jobs where theory never matches reality. So, sceptics don’t have any problem with the idea that theory doesn’t match reality. Instead we have learnt tools and techniques to focus on the key issues even when we don’t know everything.
Indeed, in most real life situations, knowing “everything” is entirely the wrong approach, because it is practically impossible and attempting to know more than is necessary to make a decision just adds time and costs.
However, academics come from a culture where theory is supposed to equal what happens. They are supposed to “know everything”, and therefore it is extremely difficult to accept they don’t.
The danger with answering the question “why doesn’t the theory match reality”, is that practically we know it doesn’t (as shown by the pause), so explaining why it doesn’t match, doesn’t add anything useful. And indeed, the right answer is “we don’t know why the theory doesn’t work as it was thought it should” … because it is important that we accept the fact the theory does not work, and instead learn how to use the information on the scale of the mismatch to help us in our predictions.
However, if I were to try to produce a long list of reasons the theory does not work, most academics will just take it as an excuse to try to dismiss each one. Then having “proved” (to themselves) that there is no rational explanation for their theory not working (based on the behaviour of those in climate) they will conclude:
THEORY = REALITY (false)
However, since I write this blog in a style that hopefully ensures no academics read it I can speculate:
- The sum of many natural variations is always a larger natural variation. Therefore when you’ve got a whole planet full of various things from living creatures, to geology to incoming solar, all changing and all adding in their own small or big way to natural variation, the total variation is huge. In other words, if a small lab experiment has noise … then as the planet is the sum of all small lab experiments AS WELL as everything else … then it include not only all the noise from all the lab experiments, but all the noise and variation from everything else.
- As a simple example of such variation, take the ocean currents. These are hugely complex and it is easy to see how they change the atmospheric temperature (as shown by El Nino). But El Nino is just one of many different currents and long term changes including the Atlantic Multidecadal Oscillation.
- Clouds …. are significantly affected by plants … and plants as we know suffer periodic droughts when they die back and then periods of huge growth. Thus cloud cover is affected by plant growth, and as clouds impact solaration and IR loss, the variation in plant growth at a regional scale directly affects global temperature. Similarly algae in the ocean.
- Solar variation & sunspots are known to be linked to climate change.
- Geothermal — we know much less about the heat flows into the oceans from geothermal than any academic would like to admit. So the idea that the surface temperature is not affected by the earth’s core is just ivory tower thinking.
- UNKNOWNs … and this is really what academics have a problem with. I include in this category things that I’ve never even considered or even knew existed. As an engineer …. I know there is a lot I do not know. As a scientist …. it’s often embarrassing to admit what I don’t know.
Nice reflections on the differing mindsets between an objective realist (like an engineer) and an academic, mostly likely a philosophical idealist–that is, one who doubts we can ever know objective reality (Heisenberg taken over the top). For such people, theory is everything. Facts are what we agree they are, and that becomes a matter of choosing your evidence.
William Briggs has summarized the objective position: Love of Theory is the root of all evil.
http://www.thegwpf.com/william-briggs-love-of-theory-is-the-root-of-all-evil/
The academics have created a theory that always requires an “expert”, themselves to interpret it. Thus they don’t really want a concise, well-understood theory. They want something that is vague, arcane. It’s job security to them. We see the same thing with Meteorology’s convection theory of storms. The theory is actually nonsense. It provide zero predictive or descriptive power. So if you want to predict or describe storms you need to hire a meteorologist. The meteorologist will completely ignore this theory and use synoptic methods, which do a pretty good job of predicting the weather. Thus, for meteorology, convection model is just a means of getting a foot in the door. The same is true for climatology. Their theory is just marketing. It’s not something they take seriously.
It’s funny, but when I first set out to tell the world of my new theory I expected people to be thankful that I was reviving an intellectually dead subject. Convection theory was so ephemeral and vague that I knew nobody would or could defend it, as has been the case. However, I never expected the depth of emotions that people have for what is such a non-starter of a theory. I now realize that this is just normal for humans. When a scientific theory is devoid of details and facts people’s minds just naturally fill-in those details with their imagination, like children do with fairy tales. And they are more emotionally attached to these created details than they would be if the details were conveyed to them by somebody or if they had read them in a book.
Unlike any of my fellow students, when I took meteorology classes I was already well educated in physics, chemistry, math and geology. So the brain-washing aspects of meteorological indoctrination didn’t have the effect on me that it had on my classmates and that it, apparently, has had on all other meteorologists. Even then I was skeptical. I had gone out of my way to take the class because I was deeply curious about severe weather. I remember sitting there as the professor explained that convection was what powered all storms. “That couldn’t be right,” I said to myself, “how could such a benign process as convection underlie the power and majesty of thunderstorms, tornadoes and hurricanes? There has got to be something more to it than just that.”
Cheers,
James McGinn
Solving Tornadoes
I would tend to put it another way. …. I was going to use the analogy of an engine and breaks … to drive fast you need both good engine and breaks … but as engineers are both good breaks and good engines.
Maybe “printer and filing cabinet”. Engineers like printers are active doing things producing lots of material … but unless you have kind of system to organise that material and store it for future use.
And I suppose I could use the analogy further … because when the filing cabinet decides its the boss and everyone else should listen to it because it has all the information. And when they start sacking the printer because the filing cabinet says it is not longer needed because the filing cabinet is the font of all knowledge … soon the whole system starts to stagnate. There’s no new paper work that needs filing or categorising … the filing cabinet then starts endless “re-organisations” and theoretical hierarchies of information … which no one is using because there is no new material relevant to most people going into it or coming out.
Coming up with new theories is easy. Off the top of my head I have:
Caterpillar thermal-tectonic theory
Pressure induced climate change theory
The “Cassandra” theory (which is far too controversial to print)
The scientifically credible alternative to “wave-particle”
Birthplace of St. Patrick
The lack of any Celts in Britain
& the Germanic origin of (some) of the early British
The problem is not finding a better theory that is more “right” than what others currently accept … it’s basically a PR campaign trying to get people aware of the idea and to slowly accept it … which takes time, effort, facts … you’ve got to give people a reason for going through the whole laborious exercise of relearning what they thought they knew … they understanding what it is the theory is trying to say … then trying to understand why the theory is wrong … then trying to understand why the new theory is right … and even then … even when people say “yes that’s a better theory” … unless they in turn are inspired to go out and tell other people … it still will not go anywhere.
Over the years, I’ve learnt that theories are just conceptual models that people have that they really don’t care if they are wrong … unless that causes a real problem to them.
So, e.g. I keep asking “is there any evidence you have … in effect that shows using the current theories does not reasonably well predict what happens.
A very good example of how a theory can be right … yet it just sits on my website is the “Caterpillar theory”. This just says that the temperature expands the crust causing plate movement. There’s nothing new here … it’s just the application of known physics.
And there is even evidence for it in the ridges at the centre of the Atlantic (which show modulations over ice-age periods). So, I’d say there is perhaps 70-90% chance of it being correct.
So, why isn’t the world biting my hand to get this theory? The answer is simple:
There’s no reason for anyone.
Having come up with the theory …and worse published it outsider academia …. academics who usually credit themselves with all discoveries … now can’t credit themselves with my discovery. Nor is it in their interest to promote this theory because it proves how they missed something pretty simple.
Nor to be frank is it in my interest. I cannot see myself becoming rich as a result … quite the reverse … I can see that the huge effort to make people aware of the theory would consume vast amounts of my time and money.
To put it bluntly neither academics nor I stand to gain commercially from this theory.
This is why academia appears to be the “font of all discoveries” … because in the past public paid academics would go out to industry, pick up ideas from people who had no interest in telling other people about them … they would then write them up as “their” idea.
And because in academia their was personal advantage to be had by “discovering” ideas academics benefited commercially through career advancement and added kudos. And (in the past) the rest of us benefited by having people who would systematically record other people’s ideas (even if the whole system was a bit corrupt – those outside USED TO benefit).
So, we created a parasite … one where people got on by taking other people’s ideas and claiming it as their own. But that beast can only work if there are people outside creating new ideas (for them to steal) and if those people themselves don’t go through the process of claiming ownership of these new ideas.
That system worked when publishing was an expensive and time consuming thing to do. Only academics had the commercial interest in going through the laborious process of getting something into print … and even if the original person who came up with the idea complained … what were they going to do? Unless they had already published the idea, they had no proof … and it was academics who wrote up the history of science … so for obvious reason we (USED TO) hear almost nothing about all the people the academic half-inched their ideas from.
In the past, the only way to get ideas accepted as your own by the academic “gatekeepers”, was to become an academic, then submit to their power and authority … wait for the people whose ideas and theories you were overthrowing to die (so that you were now top of the tree and could dictate what was “science”) … and then to publish.
For obvious reasons … given that new ideas often come from the young … and the old who champion the status quo are in charge … change within academia was measured in life-times (unless someone could come up with unequivocal evidence)
However, that has all changed with the internet. Now we have a record of those ideas before academia pinched them … and worse … through the “anti-industry” policies brought on by thge “scientific staza governmental advisers”, we have massively lost the engineering that used to be the engine of some many new ideas.
So, not only has the source of new ideas dried up, but academia can no longer so easily half-inch everyone else’s ideas and claim it as their own. So, academia is now spending more and more of its time trying to police it control of ideas and thus spending more and more of its time rejecting news ideas from outside (e.g. climate scepticism).
This is really what the climate “wars” have been about. It is a war for control of the ideas underpinning how we view climate … one which academia lost because it went down a blind alleyway on (anti-engineering) CO2.
So, one of the reasons I do not submit my ideas to academia … is because that old system whereby academia assumed control of ideas and theories can no longer be sustained in an age of the internet.
Academia can no longer thrive, living as a parasite off the ideas generated in industry & wider society (particularly when it intentionally set out to destroy UK and US engineering). Nor can it, by its control of publishing, now claim to be the source of other new ideas from outside academia (are there any new ideas from within academia?)
I’m not sure what the new world order is going to look like when academia is finally forced to admit it was never the “font of all knowledge” that it has claimed. But I’m sure the sooner we get to that situation in the UK, the better off both wider society and academia will be.
Mike:
Here is an example of what happens in the rare event that a meteorologist actually attempts to engage me in a conversation about Meteorology’s Storm Theory:
Before There Was Global Warming There Was Meteorology
https://groups.google.com/d/msg/sci.physics/3DLaGRZgxPw/8S2fMB8EBAAJ