Would be just a bit crazy since we have reference horses available to test if this is really a horse and could use a simplified test based on the traits of the reference horses. But there is no reference unicorn.
Excellent! So what do we do if there is no reference unicorn? We compile a list of expected characteristics of a unicorn, and design a list of tests to verify each of them. Per Wikipedia, we should check for
(1) goat's cloven hooves and (2) beard, (3) a lion's tail, and (4) a slender, spiral horn on its forehead.
Testing (1)-(3) is straightforward and can be accomplished by looking at the animal. (4) is a critical characteristic and should be given more attention. In particular, we should rule out the most obvious alternative hypothesis, which is that we are dealing with a horse with a narwhal horn glued to its forehead. To rule out this hypothesis, we can use the following tests:
(4a) examine the horn visually
(4b) take tissue samples from the horn and its base, examine them under the microscope
(4c) take samples from the base of the horn and run chemical analysis for traces of glue
(4d) examine the horn using X-Ray / CT
(4e) examine the entire head using X-Ray / CT to discover how the horn connects with the skull
(4f) run single-blinded comparative DNA analysis of one sample of the horn and one sample of fur
(4g) run double-blinded, multiple-lab DNA analysis of multiple tissue samples
Now, here is a little secret of experiment design:
you run the tests in order of increasing cost. This is because all results must be positive, so we can stop after the first negative and save money. For example, if at step (3) we discover that the creature has a donkey tail, not a lion's tail, then it is obviously not a unicorn, so examining the horn makes no sense. (That said, a typical scientist will probably run a few more tests out of curiosity as long as they still have budget.) The same goes for examining the horn, which we must do in detail. Again, if we find traces of glue in step (4b) or (4c), it makes no sense for us to go into the trouble of transporting the animal to the X-Ray machine.
What you are doing, is equivalent to criticizing the team after a positive DNA result in step (4f) that the trial was neither multi-site nor double blinded. Your criticism misses the point, because this is being done on purpose. If the price for a single DNA match is $1K, then step (4f) costs $1K, while step (4g) will cost on the order of $18K (3 labs, 3 pairs of samples + 3 dummies each). If you get a negative in step (4f), the you redo it just in case to eliminate lab error, if it comes out negative again, there's no point in shelling out $18K to complete (4g). As a rule of thumb, your cost will increase exponentially with each step. Looking at the animal is free; the X-Ray can be done at the local vet shop, DNA analytics costs real money.
(Do you now understand why the NASA people are testing gizmos in the air, instead of launching them to Earth-Sun L1 right away to eliminate all outside influence?)
On the other hand, if all tests (1) through (4g) come back positive then we must conclude that the animal is in fact a unicorn. And no, we are not obliged to explain how the unicorn came into being. That would take another research project -- but it makes no sense to fund it until we know that we have a unicorn an a first place!
But, observe that none of the steps (1) through (4g) is extraordinary: we are using established,
ordinary research methods to investigate the claim. In fact, any
extraordinary evidence, such as, a note saying
Yes, it's an unicorn - God would be extremely suspect. This is one reason I believe that Sagan's phrase is harmful; he demands extraordinary evidence, but then any extraordinary evidence is by definition inadmissible. The second problem with his famous adage stems from the fact, that the word
extraordinary is very ill-defined: whether the claim of having an unicorn is ordinary or not, depends on one's belief about unicorns. So the result is that one's
preconceived notion about the existence of unicorn colors her view of evidence; and that's the very opposite of how science is supposed to work.
And it gets worse. Because the notion of
extraordinary evidence leads one to incorrectly apply Occam's Razor. Let's say that I have positive results from all of the steps above. If the primary hypothesis is that we have a genuine unicorn, while the alternate hypothesis is that it's a fraud, this is the list of assumptions needed for each:
(Primary) Unicorns exists
(Alternate) Visual analysis wrong/fraud, chemical analysis wrong/fraud, X-Ray analysis wrong/fraud, DNA analysis wrong/fraud
So Occam's Razor suggests that we should side with having a genuine unicorn. But the
extraordinary claims approach allows one to assign arbitrary weights to assumptions, thus concluding without any evidence that people researching unicorns must be either incompetents or frauds; the fact that the alleged conspiracy is implausibly large is ignored, because a preconceived belief (
there are no unicorns!) outweights everything. In other words, the very notion of
extraordinary evidence introduces circular, religious reasoning into science.
Another variant of this reasoning has been advanced by Sam Harris, who abuses Bayesian statistics to essentially claim that lack of prior discovery of unicorn-like creatures renders future discovery of unicorns improbable. While this sounds common sense, like all common sense assumptions is extremely stupid. Applying his reasoning consistently one should conclude that the discovery of HIV must have been a fraud, because there were no known prior viruses infecting human T-cells. But of course Harris, like Sagan, applies this reasoning only to things
he believes do not exist.
Naomi Oreskes'
Collapse of Western civilization details how this very mindset has crippled climate research, once climate deniers started claiming the need for
extraordinary evidence and politicians followed suit.
