Tuesday, October 01, 2013

Testing Conspiracy Theories

I'm about to fly to Vienna where I'll be attending a conference on Emergent Quantum Mechanics. I'm not entirely sure why I was invited to this event, but I suspect it's got something to do with me being one of the three people on the planet who like superdeterministic hidden variables theories, more commonly known as "conspiracy theories".

Leaving aside some loopholes that are about to be closed, tests of Bell's theorem rule out local hidden variables theories. But any theorem is only as good as the assumptions that go into it, and one of these assumptions is that the experimenter can freely chose the detector settings. As you know, I don't believe in free will, so I have an issue with this. You can see though why theories in which this assumption does not hold are known as "conspiracy theories". While they are not strictly speaking ruled out, it seems that the universe must be deliberately mean to prevent the experimentalists from doing what they want, and this option is thus often not taken seriously.

But really, this is a very misleading interpretation of superdeterminism. All that superdeterminism means is that a state cannot be prepared independently of the detector settings. That's non-local of course, but it's non-local in a soft way, in the sense that it's a correlation but doesn't necessarily imply a 'spooky' action at a distance because the backwards lightcones of the detector and state (in a reasonable universe) intersect anyway.

That having been said, you might like or not like superdeterministic hidden variables theories, the real question is if there is some way to test if that's how nature works, because one can't use Bell's theorem here. After some failed attempts, I finally came up with a possible test that is almost model-independent, and it was published in my paper "Testing super-deterministic hidden variables theories".

I actually wrote this paper in the hospital when I was pregnant. The nurses kept asking me if I'm writing a book. They were quite disappointed to be drowned in elaborations on the foundations of quantum mechanics rather than hearing a vampire story. In any case, in the expectation that the readers on this blog are somewhat more sympathetic to the question whether the universe is fundamentally deterministic or not, here a brief summary of the idea.

The central difference between standard quantum mechanics and superdeterministic hidden variables theories is that in the former case two identically prepared states can give two different measurement outcomes, while in the latter case that's not possible. Unfortunately, "identically prepared" includes the hidden variables and it's difficult to identically prepare something that you can't measure. That is after all the reason why it looks indeterministic.

However, rather than trying to prepare identical states we can try to make repeated measurements on the same state. For that, take two non-commuting variables (for example the spin or polarization in two different directions) and measure them alternately. In standard quantum mechanics the measurement outcomes will be non-correlated. In a superdeterministric hidden variables theory, they'll be correlated - provided you can make a case that the hidden variables don't change in between the measurements. The figure below shows an example for an experimental setup.

A particle (electron/photon) is bounced back and forth between
two mirrors (grey bars). The blue and red bars indicate measurements
of two non-commuting variables, only one eigenvalue passes, the
other leaves the system. The quantity to measure is the average time
it takes until the particle leaves. In a superdeterministic theory,
it can be significantly longer than in standard quantum mechanics.

The provision that the hidden variables don't change is the reason why the test is only 'almost' model independent, because I made the assumptions that the hidden variables are due to the environment (the experimental setup) down to the relevant scales of the interactions taking place. This means basically if you make the system small and cool and measure quickly enough you have a chance to see the correlation between subsequent measurements. I made some estimates (see paper) and it seems possible with today's technology to make this test.

Interestingly, after I had finished a draft of the paper, Chris Fuchs sent me a reference to a 1970 article by Eugene Wigner where, in a footnote, Wigner mentions Von Neumann discussing exactly this type of experiment:
“Von Neumann often discussed the measurement of the spin component of a spin-1/2 particle in various directions. Clearly, the possibilities for the two possible outcomes of a single such measurement can be easily accounted for by hidden variables [...] However, Von Neumann felt that this is not the case for many consecutive measurements of the spin component in various different directions. The outcome of the first such measurement restricts the range of values which the hidden parameters must have had before that first measurement was undertaken. The restriction will be present also after the measurement so that the probability distribution of the hidden variables characterizing the spin will be different for particles for which the measurement gave a positive result from that of the particles for which the measurement gave a negative result. The range of the hidden variables will be further restricted in the particles for which a second measurement of the spin component, in a different direction, also gave a positive result...”
Apparently there was a longer discussion with Schrödinger following this proposal, which could be summarized with saying that the experiment cannot test generic superdeterminism, but only certain types as I already said above. If you think about it for a moment, you can never rule out generic superdeterminism anyway, so why even bother.

I'm quite looking forward to this conference, to begin with because Vienna is a beautiful city and I haven't been there for a while, but also because I'm hoping to meet some experimentalists who can tell me if I'm nuts :p

Update: Slides of my talk are here.


Phil Warnell said...

Hi Bee,

Hope you have a great time in Vienna a city long known as a gathering place for profound and extraordinary thinkers. I've had a peek at the pdf of the agenda listing the chairs and speakers and this turns out to be again one of those times I wish I was just a fly on the wall. All the days have such a stellar lineup yet besides Saturday when you talk the Sunday sessions with the Bohmians I would find most interesting. On Sunday perhaps you could ask Aephraim Steinberg what he thinks about the chances of your experiment being run in the near future. Actually in a way that would be a funny question, as if what you propose to being true is so it's either destined to be run or not.

Phillip Helbig said...

"Leaving aside some loopholes that are about to be closed, tests of Bell's theorem rule out local hidden variables theories."

Give us an executive summary on what you think about Joy Christian's "disproof of Bell's theorem".

Phillip Helbig said...

Once cliche which really is true: the best Wienerschnitzel can be found in Vienna. Definitely worth eating if you are a carnivore or omnivore.

Tobias Fritz said...

I think it's misleading to think about the assumption in Bell's theorem as the free will of the experimenter. After all, most likely that experimenter is just a physical system herself, and we can regard the announcement of her choice of measurement itself as a measurement on *her*.

If one takes this idea seriously, one finds that there are Bell-type theorems in scenarios which have several sources, and in which the independence between source and measurement setting is replaced by the independence of the sources. In fact, the need for having several measurement choices available disappears completely: it's enough for each "party" to conduct the same measurement in each run of the experiment, and quantum theory is still at odds with LHV theories.
For more detail, see this paper and this talk. (NB: shameless self-promotion!)

I am also intrigued by your proposal to test a class of super-deterministic HVTs. As you may have noticed, such an experiment could also serve as a high-precision test of quantum theory! The reason is the possible joint distributions P(A,B,A') that you can obtain in quantum theory with projective measurements are highly restricted. For example, if you consider two-outcome measurements only and look at the subset of those events in which A<>A', then the two outcomes of B occur equally often. This is true no matter what two-outcome observable B actually is!
So, conducting such an experiment might also count as a precision test of quantum theory. I tried to propose such an experiment a couple of years ago, but then got distracted. For more detail, see this paper and this talk. (Again, shameless self-promotion ;))

Sabine Hossenfelder said...

Hi Phillip,

No, I can't. I read it at some point but it's so far outside my actual research that I decided not to look into it, my day has only 24 hours. It seemed to hinge at some subtle point about what is a well-defined probability distribution, not something I'm likely to know much about. For all I know, he has made, or was at least trying to make, some testable prediction. Being a phenomenologist I'd say let's do the test and that will settle the case. Best,


Sabine Hossenfelder said...


I'm vegetarian, sorry to disappoint :p But the Austrians do well with the sweeter pieces of bakery. They could learn something about bread from the Germans though. Best,


Sabine Hossenfelder said...

Hi Tobias,

Thanks for these references, will have a look! Will you be at the conference in Vienna? Best,


Sabine Hossenfelder said...

Hi Phil,

Well, yes, the future is set. As I elaborated on in my earlier post though that doesn't mean nothing has to be done. Indeed, Vienna is not only a beautiful city but also historically interesting. Not to mention that one can do some serious shopping ;) Best,


Phillip Helbig said...

Vienna also has a cheap (at least compared to Frankfurt; Stockholm used to be cheap but I don't know how it is now) weekly pass for public transportation (which covers an area and population much larger than Frankfurt).

Michael Gogins said...

I hope you get some good feedback on your idea and, if it is vetted, that the test is actually done, and more than once. I'm not a scientist, but I do think that the foundations of quantum mechanics are some of the most important science we have done and that those foundations must be probed as deeply as we can go.

I don't think "freely chosen" means the same thing in experiments as it does in philosophy.

I think in experiments it means that the theory would hold if the experiments were performed with certified random settings. And I think that science has, so far, made this a fundamental requirement for theories.

I think in philosophy it means many things. Many think that the kind of randomness in the paragraph above is required for human free choice. But arguably, human free choice is compatible with determinism. In this view freedom means that nobody is forcing or fooling you into making your choice, not even your own body and unconscious, you are being really rational.

Anyway, I think your thought is a good one and, again, I hope that it will be tested.

L. Edgar Otto said...

Yes! This. Quasi local delay would show super GR determism too A brillant thought or experiment that grounds QM bottom up intelligibly in useful info locality. SUPERDUPER hidden. Or not!

Uncle Al said...

" in the former case two identically prepared states can give two different measurement outcomes, while in the latter case that's not possible. Unfortunately, "identically prepared" includes the hidden variables and it's difficult to identically prepare something that you can't measure. Construct a large quantum calculation with spins as the qubits (e.g., implanted nitrogen centers in diamond). The outcome is determined either way. Consider the two cases' contrasted noise spectra.

A C_60 molecular beam multiple slit diffracts, C_60 appearing on the other side, even in the summed sparse case of single molecules arriving. A D_3-trishomocubane molecule has six indistinguishable homochiral centers of eleven skeletal carbon atoms. Rigid skeleton. If a molecular beam of enantiomerically pure molecules multiple slit diffracts, is the other side chiral or racemic, even versus odd number of slits?

Theophanes Raptis said...

Well, to say it in simple terms, not only you 're not nuts but you are endangering yourself with a knowledge that you may not be able to bear. For it is obvious that every "robot" without a free will cannot control the ways misfortune could be applied unto it by those that possess such a quality...

Tobias Fritz said...

@Bee: no, I won't be in Vienna, I just learned about this conference from your post! But if you happen to take a look at the papers/slides and have any questions, I'll be happy to discuss by email.

YourGodFearsMe said...

So... when are you going to write that vampire story? :P

Don Foster said...

Dr. Bee

It would be educational to hear, roughly step by step, the path one must navigate between having an idea for an experiment and actually having it performed.


Uncle Al said...

@ Don Foster (It's a cecropia moth.) Experimental physics requires infrastructure and fineness of its weave. You must convince a disbelieving somebody else - and its business plan, and administrators whispering "resource embezzlement"- to support it. Novel = risk ("no").

Discovering proton nuclear magnetic resonance (NMR) used a block of paraffin and an impressive Helmholz pair of electromagnets, among other stuff. It was carefully calculated, planned, executed, and...NOTHING. Two guys diddled, extrapolated, and agonized for hours, for days. Nothing at all. At the very end, one of guys shouted an expletive and violently spun the magnets' rheostat dial. The soft iron cores were saturated. The extra current increased the field as the two of them watched the signal crawl across an oscilloscope. Turn it down and the signal marched back.

Hire experimentalists you would never hire. The universe requires a good swift kick to transition. After that, send in accountants (and fire the guy who damaged the rheostat).

Eric said...
This comment has been removed by the author.
Eric said...

One thing that would be worth researching is if there is an internal mechanistic structure within all fermions that accounts for changes in kinetic energy during acceleration. That is, is it just a given or can it be like an electric motor that increases [decreases] its angular momentum during acceleration[decelleration].

It there was/is such a mechanism then it would pretty neatly account for entanglement. If any increase/decrease in J was inhibited during acceleration/ deceleration then the resulting field disturbance would be a quantum wormhole. In other word, if a particle does not change its kinetic energy during acceleration then the quantum field must instead be molded along that path to the constant energy of that particle. It would look (and be) a wormhole in all respects because the energy used for acceleration would be used to organize the field along a linear path.

Sabine Hossenfelder said...


"It would be educational to hear, roughly step by step, the path one must navigate between having an idea for an experiment and actually having it performed."

Yes, in fact. If you come across a step by step instruction, let me know, I'd be interested to hear. Best,


Don Foster said...

“Yes, in fact. If you come across a step by step instruction, let me know, I'd be interested to hear.”

Well, that is surprising. So many having come so far it would seem there would be some kind of guidebook.

Perhaps advertising --maybe a good PR firm would help, some trendy commercials during soccer games.

Or you could take to wearing a sandwich board around at conferences; have a petition for folks to sign.

Of course, since it’s a conspiracy, you may need to find a more subversive route.

In any case, a heartfelt viel glück!

vmarko said...

Hi Bee,

As far as superdeterminism goes, I still fail to understand how can one claim that it isn't a conspiracy-type theory? I understand that Bell's theorem does not hold in this setting, but wouldn't it be more natural for a deterministic universe to actually behave like one, and not violate Bell's inequalities to begin with?

On a more serious note, my feeling is that superdeterminism is "cognitively unstable" (a term I borrowed from Sean Carroll's blog). If the experimentalist does not have "free will" to explore the whole phase space when setting up initial conditions for an experiment, how can one discover any laws of physics at all? // On a slightly philosophical note, the absence of free will denies one the capability to learn, and the concept of knowledge becomes sort-of fuzzy IMO. //

As for the experiment to test the superdeterministic behavior, I admire the idea, it looks promising, and I really hope you can get someone to perform the actual experiment. But I don't quite follow how can that experiment test for superdeterminism, even a subclass of it. Namely, if superdeterminism is true, than any result you get from that experiment is a consequence of the past correlations between the apparatus and the non-free will of the experimenter setting it up.

In short, if all experiments are assumed to be a priori biased for some result, you cannot trust them. And if you cannot trust experiments, science is impossible.

I'd really appreciate if you could help me understand better how superdeterminism can be considered a reasonable point of view, in the above sense.

Best, :-)

Eric said...

Bee, I think you might be leaving out a significant distinction when talking about the "inertia" of a repeatable experiment. (My description not yours.) Energy is always required to both measure and prepare experiments. In the case of non-local experiments there is a significant difference in the preparation energy input between bosons (photons, not W and Z and gluons)and fermions.

For instance, in the case of long distance non-local correlations, like photons traveling through fiber optics, very little if any energy is required in the act of separation of the two photons from one initial photon. The photons remain correlated because the fiber optic cable inhibits environmental friction that would destroy the correlation. Without the fiber cable the photon correlation will be destroyed immediately upon measurement. That is the measurement will be correlated but any later correlation is destroyed. The key in both situations is that there is not energy put into the acceleration of the photons away from each other because photons have zero rest mass.

On the other hand two fermions that have been together and had a "local" sharing of opposite spin, but which have then been separated through acceleration, will have energy input within that path of separation. That is because fermions have internal mass and require energy to separate them. But if you correlate them as you accelerate them away from each other then it seems to me it is the same as saying you are holding that spin state relationship constant. Again, it might be possible to think of this as holding the absolute spin of both opposite spinning fermions constant.

This would be different from the normal concept of just holding the "relationship" of spin between the two fermions constant. If spin normally changes during acceleration, as I suspect it does, then the energy involved in accelerating the fermions apart would instead create a somewhat stable micro wormhole between the two.

You might have more luck creating a repeatable entanglement experiment with fermions, at least in this respect. But you would probably have to put substantial energy into creating these separation wormholes before the entanglement effects become measureably stable.

Eric said...
This comment has been removed by the author.
Eric said...

"Without the fiber cable the photon correlation will be destroyed immediately upon measurement."

Actually I meant that the fiber optic cable just preserves the correlation longer. Any kind of photon entanglement will be immediately destroyed after measurement.

I should add that in both fermions and photons different measurements of spin orientation would devolve entirely to the measurement apparatus and local conditions where it is located. They would be correlated between two particles but could not be predicted beforehand and could not be predicted in a later measurement and would not be correlated.

In the case of a somewhat stable micro wormhole between fermions you still could not predict the spin orientation beforehand as it would depend on the measuring apparatus and local conditions. But it might be reasonable to predict within a certain statistical error what the spin orientation would be during the second measurement of the two particles based on the first measurement.

Sabine Hossenfelder said...

Peter Vilters:

You sent me an email regarding this blogpost. My reply was rejected by your server as likely spam. I don't have the patience to figure out why that is. If you want a reply, either post a comment here, fix your spam filter, or use a different provider. Btw, I don't particularly like it if people send me comments to my blogpost by email because this means that by all chance I'll have to repeat the reply. Best,


Sabine Hossenfelder said...


I don't believe that what you or I find 'natural' has any relevance for the laws of nature. I don't understand your problem. Just think of it as a boundary condition. The absence of free will of course does not mean that one cannot learn anything. Please read my earlier blogpost. You, as a subsystem of the universe, can of course still gather and acquire information that you previously did not have: you learn. Best,


vmarko said...

Hi Bee,

My problem with superdeterminism can be neatly formulated along the lines of Anton Zeilinger's words here:


In short, if superdeterminism is true, the knowledge one obtains by performing experiments cannot be trusted. I am not saying that one cannot learn, but that the quality of what has been learned is a priori not trustworthy. If the initial conditions are such that nature is fooling us into thinking that Bell's inequalities are violated (whereas they actually are not, given that universe is deterministic), then there might also be other things we claim to know about nature that are also wrong, because our non-free will makes biased experiments.

Given that, it seems to me that superdeterminism naturally leads one to solipsism, and the denial of any knowledge about the external world (i.e. nature). I don't see how can that be a feasible starting point for doing any science.

I usually imagine superdeterminism as the execution of a computer algorithm, where initial data is built-in and cannot be changed. A "person" is a data structure inside the algorithm, experiencing virtual reality. But the algorithm can be such that this "reality" can have arbitrary laws of physics, even inconsistencies and paradoxes, whereas the "person" is rigged (through initial data) to never observe any paradoxes, and instead thinks that the external world is consistent and well-behaved (contrary to objective reality of that world). The "knowledge" that this person accumulates is not real knowledge about their reality, but only a convenient set of prejudices, creating an illusion that they observe a self-consistent external "world" with certain properties.

If one assumes superdeterminism in the real world, there is no difference between the quality of "real-world" knowledge we obtain through experiments, and the set of prejudices that the virtual person has about their virtual reality. They are both biased and not trustworthy. So I don't see how can one do science in such a setup?

Best, :-)

Amos said...
This comment has been removed by the author.
Sabine Hossenfelder said...


"I usually imagine superdeterminism as the execution of a computer algorithm, where initial data is built-in and cannot be changed"

That's determinism. You're missing the essential point of superdeterminism which is spatial correlations.

"the knowledge one obtains by performing experiments cannot be trusted"

That's nonsense of course, as any viable superdeterministic theory must reproduced standard QM in the appropriate limits (typically when averaged over hidden variables). And besides this my paper points out an additional knowledge gain outside the already tested limits.

I don't understand your philosophical issues here and I find them quite useless. Look, the question I've asked myself is this. It's a possible description of nature yet Bell's inequalities can't be used to test this possibility, so what can we do to test it? That's the question I've tried to answer and the above explained experiment is my answer. Best,


Cynthia said...

Keep in mind, Bee, a "conspiracy theory" is a mental model for sorting out the truth. It is like the proverbial finger pointing at the moon. Don't obsess over the finger. It is the moon you should be looking at.

Kingsley Jones said...

I think it is good to see such discussion once more about a possible deeper layer of determinism (or not). It cannot hurt to concoct new forms of experiment and test them.

The one thought I would plant as a seed, which seems not much discussed, is the idea of Psi itself as a hidden variable.

At first blush, this will seem a little foolish. However, that is because people are used to the idea that Psi describes the probability of finding point-like particles here or there.

If you think that way then you probably should not be even worrying with the topic of this post. You presumably know it is misguided.

However, if you think the Copenhagen interpretation may be misguided then you are entitled to think of Psi as the state, which is the hidden layer of determinism, but that it is not directly observable.

You can develop a theory along these lines (I did one in 1995), but I am sure there are others.

I mention this because much of the contemporary discussion seems to be a bit stuck in a rut.

People reject Copenhagen but they want point particles.

It seems to me, that if you know the history of quantum physics, this half-way position is untenable.

If Psi is hidden then it must determine something that is not hidden but which is not deterministic.

In my 1995 paper I made a suggestion.

It may not be the only one possible.

Eric said...

Hi Kingsley,
Psi (the wavefunction) may be a bit too abstract for those of us who are not mathematicians. (I googled you.) I think what mathematicians do is extremely important, especially in physics. But I think its important to always remember that math a is tool for calculating results rather than physical reality. For instance, vector angular momentum is always perpendicular to the spin plane. I doubt that anyone would argue that physical angular momentum effects that are visible are all within the spin plane. However the cross product of forces with the vector angular momentum predicts the torque magnitude and direction. Its always important to differentiate the math used to predict things from the physical object.

What I'm saying is that entanglement (to me) involves a physical effect than can be visualized just like physical angular momentum. But it cannot alone be used to make predictions and calculate stuff. You need psi for that. But personally I don't think that psi is a real thing but instead is an abstraction of a real thing that is just useful for making mathematical predictions.

I believe angular momentum in subatomic particles is real and physical and there is a reason that two fermions that are entangled act like a boson. The reason is that two fermions that are entangled have their two spin planes held parallel. This is not normal behavior for a fermion. Normally the spin plane of fermions have a torque induced when a force is applied to them that will tend to bring the spin plane in a direction parallel with the direction of the force.

I think what is happening is that in the instant of separation all the force is applied when the spin planes of the two particles are parallel and entwined. Since the energy of acceleration does not have time (it happens instanteously) to torque them into a position with the spin plane parallel to the force separating them that force is instead used to organize the quantum field to the constant kinetic energy of each individual photon or fermion.

What seems to me to be happening is that people are being confused by the decay of this quantum wormhole caused by measurement. Since the wormhole only has energy input at the point of location of the initial local sharing of spin there is no way there is enough energy to make that micro wormhole stable. Measurement entails energy and just like a tiny dust devil you step into the very act of stepping into it destroys it. Psi just is a useful tool for descrbing this mathematically but it does not get close to what is a physical deterministic effect.

Eric said...

"I doubt that anyone would argue that physical angular momentum effects that are visible are all within the spin plane."

That may not have been clear. What I meant was that all angular momentum effects that are visible are within the spin plane and most people, even mathematicians, would probably not argue with that.

Nada said...

Dear Hossenfelder,

I have been fond of the idea of a sub quantum theory for a long while, but I wonder what is your motivation for rejecting the Everettian reading ?
It seems that a lot of people have adopted it after David Wallace and the rest of the "Oxford Everettians" claimed the preferred basis problem and Born Rule problem solved.

Ulla said...

Then you’re stuck not only with Bell’s inequalities, but more generally with the whole quantum picture of reality. So, I think you have to assume that Bob has made a decision not out of free will, but by some predetermined correlation. Gerard T Hooft in sci am intervju.

Impact from future instead of free will? :)

Rodney Bartlett said...

I couldn't post everything I want to say (it's too long). So I'll just post the abstract, together with a link to my short article (for those who are interested).

Abstract -
Beginning with the Moon’s reflection in water, that reflection is then compared to physicist David Bohm’s holographic universe and holographic brain, and merged with Albert Einstein’s three universes in one cosmos. This results in Professor Max Tegmark’s hypothesis of mathematical formulas creating reality, and the maths is converted into the physical reality of an infinite, steady-state universe made up of finite, “bubble” or “pocket” subuniverses that originate with big bangs (we live in one of these, and the conversion is achieved via “digital” string theory). It’s concluded early on that all these subuniverses contain a 5th-dimensional hyperspace that allows time travel into the past, and a webpage by Dr Adam Riess (2011 Nobel prize in physics) is used to show how this hyperspace results in dark energy and dark matter. Along the way; Einstein’s Unified Field is justified (with Professor Penrose’s error being exposed), gravity is referred to as a repulsive force that causes attraction by pushing a falling apple to the ground (Isaac Newton’s mathematical description remains intact), dark matter’s role as the scaffold for normal matter is interpreted via fractal geometry, and the four forces are accounted for in a new way. The article is presented in the readable style of plain English, leaving the mathematics and scientific language to the above-mentioned scientists. I wonder if the Large Hadron Collider can achieve sufficiently high energies to observe phenomena that will give some support to the existence of another dimension, and therefore of the other concepts in this article.

Link -

Kaleberg said...

Back in the 1970s there was a lot of excitement about the wonders of the new RSA and other related public key cryptography systems, but then a friend of mine simply asked "Yes, you can encrypt wonderfully if you have a set of large prime numbers, but where did you get your prime numbers?" Reading this post and thinking about superdeterminism I keep hearing his voice asking, "Where do you get your entangled particles? Where do you get them into your detector? How do you decide how to set your detector?"

No, I'm accusing the NSA or anything, but in quantum physics, as with cryptography, a certain paranoia is in order.