Extraordinary Claims Require Extraordinary Evidence

In the particle physics world, there is a lot of excitement today as the MiniBooNE experiment just released a new paper looking for what happens when neutrinos (from a muon) turn into neutrinos (from an electron). And particularly how often that happens.

The MiniBooNE experiment at Fermilab uses the accelerators at Fermi National Accelerator Laboratory to make muons that then turn into neutrinos. Then they look at those neutrinos a bit further away, about 500 metres. Neutrinos don't do much most of the time so it is easy to send them for 500 meters or further, through rock, it does not affect them as they only interact very very rarely.

MiniBooNE can measure neutrinos well and does that in many ways, in this case it published a paper that describes a long-awaited analysis that measures the appearance of electron neutrinos by identifying the photons that are created when an electron neutrino (extremely rarely) interacts in the tank filled with oil they use as a detector. This neutrino will create an electron, and that electron will create a flash of light, a photon. The tank is filled with light detectors to see those flashes of light.

This is an excellent way to identify particles, and the experiment has already been rather successful at seeing the neutrinos from muons that are being aimed at the oil tank from the Fermi National Accelerator Lab complex (in that case you would see a muon in your oil tank). Obviously they’re set up to do this and they are very good at identifying these particles, shooting muon neutrinos at an oil tank and then measuring the flashes of light created when something happens. The measurement that came out yesterday is the one where instead of muons, the detector identifies electrons, and then specifically the light that is created when the electron flies through the oil. And what you in that case measure is photons, and they know how to do this. Up to there it is all great, and seeing electron neutrinos is not only interesting (we know neutrino oscillations exist, see 2015 Nobel Prize ), but also really worthwhile doing.

Now you get to the real details of the analysis: there are other particles that produce photons, so just seeing photons is not enough, you also need to know how many came from something else, particularly from some other particle creating a similar signature. One of those particles is a neutral pion (a particle made of two quarks), which can be created when a muon neutrino hits an atom and knocks some of the quarks out of the protons and neutrons. This is extremely difficult to predict as neutrinos interacting with neutrons and protons or atoms or nuclei in general is very very difficult to calculate. However, MiniBooNE expects that about 1/3 of their photons are actually due to these pions, not due to appearance of electron neutrinos. It is their largest background and rather difficult to calculate or predict, so they use a trick and predict it by measuring the case where the pions can be identified, so essentially saying: if we see that many pions this way, we should be able to get the number of photons from pions in our analysis. All reasonable, except that this prediction (which is called an in-situ measurement) is still not so easy. It is difficult to predict the number of these photons from pions, so MiniBooNE does the scientific thing and assigns an uncertainty to their prediction, they say that they can predict the number of photons from pions within 13% or so. The problem is that MiniBooNE makes really strong claims after that, specifically that the chance that they see extra neutrinos and that the chance that those are fake or background is one in 50 million, or 4.8 standard deviations in particle physics jargon.

And now we get to the interesting discussion: the excess that MiniBooNE sees and assigns to extra electron neutrinos (beyond the ones they anticipated and beyond the background they expected including the one from the pions) has the same energy distribution as those photons from the pions. So it behaves the same, has the same shape, both contributions are higher at low photon energies and then fall off exponentially. I think that one in 50 million chance of being wrong is in that case a very bold claim. Extraordinary claims require extraordinary evidence, and I am not convinced.

Being not on the MiniBooNE collaboration it is very difficult for me to check if this is the case, particularly with a paper in Physical Review Letters, a journal that restricts the number of pages to authors. I just pointed out that all the ‘extra’ photons assigned from neutrinos behave exactly like the ‘background’ photons from pions, and this is also supported by the few extra plots the MiniBooNE collaboration provides besides the paper. So in principle, if that pion prediction was an underestimation, or in principle you could even just underestimate the uncertainty to do this, you could explain most if not all of that excess away with photons from pions.

So no, I would definitely not put money on that excess being real, independent on the number of standard deviations that they quote. Because standard deviations only become relevant when it is clear the background is understood and I am not a 100% convinced there, unfortunately. Note that I am not competing with MiniBooNE, I would really appreciate this to be true but as a scientist my first role is to question that the result is convincing and as said, I’m not 100% convinced yet, more like 50%. And definitely not 1 in 50 million convinced ;-)

Still, it is an extremely exciting result, I really hope that this will help to understand the neutrino mystery, which is one of the biggest questions we currently have in fundamental science. There are multiple experiments with contradicting results, so the final answer is definitely not out. It is an enervating time to be a particle physicist, new results are appearing (and regularly disappearing, such is the scientific method) very frequently and I hope we will very soon get a consistent picture where multiple experiments can confirm or eliminate this excess! There are experiments in preparation that are much better at removing this important and difficult background, and MicroBooNE's first results ( and later SBND/ICARUS )
will remove any doubt whether the excess is true. Some years of patience more ...