14

Good Science, Bad Science

Image

One of the surprises that greeted me when I first plunged into the murky waters of cryptozoology was its relationship with mainstream science. In past decades people like Bernard Heuvelmans, John Napier of the Smithsonian Institution, Grover L. Krantz of Washington State University and others had spent a great deal of time trying to identify the creatures they were dealing with. These men were trained as scientists and by and large kept to the principles of their calling. They may not have succeeded in providing the key proofs that would satisfy a critical audience, but at least they were trying their best to do so. That much is clear, particularly in the case of Bernard Heuvelmans, as was very evident from his correspondence in the Lausanne archive. He may have enjoyed the celebrity that surrounded him, but in the quiet of his own study where he catalogued every press report and every letter on his subject, systematically marking each one with his red felt-tip pen, he was clearly a serious scholar. I saw no hint in his archive that he was anything less than honest. He may, on occasions, have been less critical than he might have been, but there was no sign of deception or fabrication. He worked only slightly beyond the orbits of full-time professional scientists, like Napier, but he was on good terms with them and had their respect. At least he did until he was undone by the Minnesota Iceman.

Heuvelmans did not do any laboratory research himself, but was in touch with scientists who did have the facilities to work on material brought back from the wild. Unlike Heuvelmans, who devoted most of his life to cryptozoology, these scientists had plenty of other things to occupy them. Without proper financial support they were only willing to give a portion of their time to these fringe cryptozoology projects, and rightly so, as the income from these projects never covered their expenses. Aside from the occasional sponsorship of wealthy men like Tom Slick and his associate Kirk Johnson Sr, the scientists who worked cryptid material did so with no financial support. The only other way of getting the necessary funds to pay for the research was through writing books and articles, and appearing on the radio or TV.

Of course, there is a price to pay to please the avidum genus auricularum, the enduring mass of David Hume's gazing populace, which any media outlet must amuse and entertain and, if necessary, inform. While the ‘inform’ aspect of reportage might take priority in the coverage of major political or financial events, it is the ‘amuse and entertain’ potential of cryptid stories that appeals – not necessarily the ideal foundation for proper scientific enquiry. That is not to say that the ‘amuse and entertain’ potential of scientific research should be overlooked. I have thought for a long time that scientists do not pay enough attention to the appeal of their research to the general public who, after all, are very often their paymasters and by whom they are entrusted to work on their behalf. But support is quickly withdrawn from any scientist who deliberately invents his or her results. For a professional scientist to be found guilty of fabrication is the end of the line. They face dismissal, and sometime prosecution. The integrity of scientists is absolutely essential for their world to survive and prosper. All professional scientists understand this and ruthlessly pursue and eliminate transgressors.

The way professional standards are enforced, or at least encouraged, is by publication of results in scientific journals after close scrutiny by other scientists in a process known as peer review. In other walks of life that might seem an unsatisfactory way to ensure honesty, and perhaps it is. The peer-review system is not primarily there to root out corrupt and dishonest practices. It is there to ensure that the experiments have been properly carried out, that there is enough detail for them to be repeated and, most importantly, that the conclusions are supported by the evidence. Without peer-reviewed publication it could be argued that a piece of research is not really science at all.

That can be a harsh conclusion. Some of my own work has not been through this process yet I still consider it valid science. I have done plenty of experiments that produced negative results, and not bothered to write them up for publication. This is the fate shared by much of the cryptid work done by professional scientists. Hardly any of it has been through the peer-review system and often for very good reasons. Like the press, scientific journals prefer novelty in what they publish. The key difference between the two is that, while the media are able to, and often do, report a newsworthy project when it is in the planning stage, a scientific journal needs evidence, which can only come after the research is complete. Take, for example, the case of Oliver the chimpanzee's crucial chromosome count. The work had been done but the results were not written up for a scientific journal. The popular press, while keen to write about the proposed testing in advance, lost interest in reporting the result when it showed that Oliver was just an ordinary chimp.

I can give you an example from my own laboratory. As I have already mentioned, in 2000 I received three samples from the Himalayan Kingdom of Bhutan that were attributed to the migoi, the Bhutanese equivalent of the yeti. The genetic investigation was to be part of a documentary film about the migoi being made by Harry Marshall of Icon Films. (It was also Harry's company that made ‘The Bigfoot Files’ which followed the ‘Yeti Enigma’ project.) All three of the migoi samples that reached the lab had compelling provenances. The first was from a skin kept in a monastery that was used to treat sick monks. A night spent sleeping on the skin of the migoi was enough to dispel almost any malady, apparently. The second sample came from a tree where ‘the migoi scratches its skin’ while the third was found in a migoi nest inside a hollow tree to which the film crew was directed by the King of Bhutan's migoi steward. Like the swans of England, all Bhutanese migois are royal property.

After a frustrating few months of trial and error, my research student Helen Chandler managed to get some DNA from two of the three samples and identified the species through their sequences. The skin with remarkable healing properties came from the brown bear Ursus arctos, while the hair left in the traditional scratching tree was from an Asiatic black bear Ursus thibetanus. The third sample, a single hair from the hollow tree, we could not identify. When the film was broadcast as ‘Wildman: Hunt for the Yeti’ there was only a brief mention of the bear identifications, but a good deal about the unidentified third hair. Indeed in the closing frames of the film I am heard to say words to the effect that: ‘Normally we would have no trouble identifying a good DNA sample. But then we have not dealt with the yeti before.’ A very hammy performance I thought, when I saw it again recently.

Of course, it is perfectly understandable that, as a film-maker, Harry would want to end the film about yetis on a note of mystery, but the point is that while we could have written up the bear results, no journal would even consider publishing what we found out about the mystery hair, which was absolutely nothing. Just like Dr Ledbetter, we ‘never got around’ to sending the bear results to a journal, though it did produce a chapter in Helen's thesis – which is at least in the public domain in the sense that there is a copy lodged somewhere in the vaults of the Bodleian Library in Oxford. The hair that we were unable to identify soon transmuted into the hair of ‘an unidentified species’ and became something of a celebrity in the shadow world of cryptozoology. I don't think I have ever had as many enquiries about any of my work as about that third hair. But as you will read later, I recently tested what remained of the mystery hair and made a positive, and very surprising, identification.

When I set out on this current project, I was frustrated that so many press reports of yeti and Bigfoot samples that were ‘on their way to be DNA tested’ were never followed up. The results of these tests were seldom, if ever, reported. More cases of ‘I didn't get around to it’, I suspected, so I made it my business to ask some of these scientists if they could tell me what they had found and, if possible, let me have a look at the data.

What I found was a rather sorry state of confusion. Samples that I knew had been sent to good labs by enthusiasts were often lost or the results not relayed back to the donors. Material that had been carefully collected and submitted with high expectations was often treated with casual disdain. Being on the receiving end of unsolicited sample donations, I am sometimes guilty of not acknowledging receipt right away or not being quick to return results. Cryptozoologists have sent samples for various forms of DNA testing to my lab and others, but rarely consider who is going to pay for it. Unsurprisingly, a sample that arrives uninvited and unaccompanied by the means to pay for the analysis is not always given the priority the donor considers it deserves.

You may now understand why lab results, properly written up, are such a rarity in the cryptid world. That doesn't stop references to genetic testing in the many books written around the subject. For example, in 2000 hair was recovered from the site in southern Washington where a Bigfoot was alleged to have rested on soft ground, leaving a much-studied impression: the famous Skookum cast. This hair was sent for analysis to Dr Craig Newton at the University of British Columbia. This is referred to in Dr Jeff Meldrum's book Sasquatch: Legend Meets Science and tells us that Dr Newton chose to analyse nuclear rather than mitochondrial DNA.1 His conclusions, that he was unable to rule out human contamination, are quoted by Dr Meldrum but there is no reference to the data, which I presumed Dr Newton had ‘never got around’ to publishing. So I tracked him down through colleagues at the University of British Columbia and this is what he told me:

‘At the time I was working on Kermode bear genetics (Kermode bears live on the coasts of Alaska and British Columbia). I tried my universal melanocortin 1 receptor primers (like the mitochondrial 12S RNA gene that I was using, able to identify species origin) on one or two of the Skookum cast hairs but only really managed to amplify myself. The amplifications (the process of increasing the amount of DNA in the test-tube until there is enough to sequence) were pushed to the limit and the DNA sequence was bad, so I didn't really believe anything, hence no publications or anything like that. The problem was old degraded hair samples to start with and targeting a nuclear gene with low copy number per sample. The only sequence I could ever read well was my own. The idea at the time was to be able to rule out false sightings – bear, moose, hairy humans etc. – assuming we had reasonably good samples to start with. But I've seen all sorts of things: apple core chewings, peanut husks, unusual turds, saliva smears off windows, bloody hair masses – but to this point nothing I've been able to get good data from. I am a bit ashamed, but then nobody was writing cheques and there is only so much effort that can be spent on such whimsies.’

I have found only two examples where the results of DNA analysis have been published in the scientific press. In 2004 Michel Milinkovitch2 and his colleagues claimed to have tested a clump of hair brought back from Nepal by Peter Matthiessen, the explorer and author of The Snow Leopard.3 The DNA results showed it to be most likely from an ungulate (a hoofed animal) of some sort, though the species was not identified. My suspicions about this paper were raised when I could find no trace of the sequence in the database into which, according to the paper, it had been accessioned. Suspicion mounted when I read that their findings confirmed Captain Haddock's hypothesis, elucidated in Hergé's Tintin in Tibet, when Haddock addressed the yeti as ‘You odd-toed ungu-late.’ Only then did I notice the footnote that ‘. . . evolutionary biologists need to retain a sense of humour’ and the publication date: 1 April. Fortunately, as it turned out, I missed the footnote on first reading and had already contacted Prof. Milinkovitch at the University of Geneva, where he runs the Laboratory of Artificial and Natural Evolution, to enquire about this research. After introducing myself and my current project I asked about the Matthiessen sample:

I recently came across a reference to your 2004 paper in Mol. Phylo.Evol in which you sequenced the mitochondrial 12S RNA gene from a sample brought back from Nepal by Peter Matthiessen and attributed to a mehti (a local name for the yeti). I assumed this paper to be a good joke, as I could not find any yeti sequences in the EMBL database and you did not give accession numbers.

Of course it is reassuring to have Captain Haddock's preliminary results on the identity of the creature confirmed by DNA sequencing, and (had I located the sequence) I would have seen whether EMBL or GenBank could provide even closer identification. As your paper has been referenced in all seriousness by other workers, I just thought I should double-check that it was genuine.

When I read the footnote in the journal, I immediately emailed Prof. Milinkovitch to say that now I knew it was a practical joke, there was no need for him to reply. But reply he did, which is when I got the whole story. Prof. Milinkovitch really had tested the mehti hair brought back by Matthiessen. It had arrived in the lab because Matthiessen had known the distinguished evolutionary geneticist Dr George Amato, now the Director of the Sackler Institute for Comparative Genomics at the American Museum of Natural History in New York, and had asked for his help in doing a DNA analysis on the mehti hair sample.

As Matthiessen describes in The Snow Leopard, he was on an expedition to the Himalayas in 1992 when he and his companions came upon some unusual footprints in the snow. These were immediately identified by his guides as the prints of a mehti. A skein of twisted hair was found beside a river at the bottom of a nearby gorge and this was also identified as belonging to a mehti by their Nepalese guides.

At first Prof. Milinkovitch thought that Matthiessen was pulling their legs. But he was deadly serious. He was convinced he had a genuine mehti specimen. Correctly, Matthiessen insisted that the identification of any new species would have to be reported to the Nepalese government before publication, a request with which Milinkovitch and Amato were happy to comply. In the event, and to Matthiessen's bitter disappointment, the DNA analysis clearly showed that the mehti hair sample was actually from a horse, Equus caballus. The published paper did not reveal the species identity, only that it was an ungulate and that this confirmed Captain Haddock's earlier conclusion on the identity of the yeti.

As Prof. Milinkovitch added to his reply: ‘I am Belgian, and as any real Belgian guy, I know Tintin au Tibet by heart.’

The mehti sequence, though declined by GenBank, is publicly accessible through the professor's laboratory website.4 I ran a 150 base segment through a GenBank search to see if there were other species identifications. Ninety-eight of the first hundred sequence matches came back as Equus caballus, the horse. However, the last two were matches with the kiang and the onager respectively. Both are wild asses. While the onager is widespread in India, Pakistan and the Middle East, the kiang is found in Tibet and occasionally over the border in Ladakh, India and northern Nepal. Although Matthiessen's mehti was probably just an ordinary horse, it might just have been a kiang. It most certainly was not a primate.

The second example, published in the respected Trends in Ecology and Evolution, reported DNA results from a sasquatch sighting in the town of Teslin, Yukon.5 Nine residents witnessed a large biped moving through the brush outside the kitchen window of their cabin. They were convinced they had seen a sasquatch and, when they found a tuft of coarse, dark hair very close to a large footprint seventeen inches long and five inches wide, they sent the hair to the Government of Yukon Department of Environment for identification. From its microscopic appearance the lab thought the hair was probably from an ungulate, maybe a bison, and passed it on to the University of Alberta for DNA testing. This confirmed the visual assessment that the hair was indeed from a large ungulate and probably a bison, and concluded that this was what had fooled the good people of Teslin.

As with the Milinkovitch mehti paper, I emailed the principal author, Dr David Coltman from the University of Alberta, to ask whether the article was genuine. My enquiry produced more of the back story and Dr Coltman told me how he had become involved:

My attention was attracted because the regional biologist from the Yukon, Philip Merchant, stated that the hair closely resembled bison. I also happened to know Philip quite well and he is a very competent and diligent professional. But the question about whether the hair was really bison or not remained, so I called Philip and asked him to send it me and we would DNA test it so we could bring something definitive back to the press, but really for a bit of fun, but also because I saw this as a good opportunity to do some good outreach about science and wildlife genetics.

When the sample arrived, Dr Coltman successfully extracted and sequenced the mDNA, in fact the same control region that we have already encountered, and detected a clear match with bison. The story made headline news with a campus press conference, interviews on CNN and coverage by all the major TV networks. Soon afterwards he was having dinner with the editor of the influential Trends in Ecology and Evolution and she persuaded him to write a short article. Rather like Prof. Milinkovitch's paper on the Matthiessen mehti, Dr Coltman could not resist a titillating final paragraph:

There are several possible explanations for these results. First, as suggested from molecular analysis of hair from a suspected Yeti (referring here to the Milinkovitch paper), the sasquatch might be a highly elusive ungulate that exhibits surprising morphological convergence with primates. Alternately, the hair might have originated from a real bison and be unrelated to the sasquatch. Parsimony would favor the second interpretation, in which case, the identity and taxonomy of this enigmatic and elusive creature remains a mystery.

Like all the best yeti and sasquatch tales, it ends with an element of uncertainty. Normally this ambiguity is genuine – indeed the extent of the uncertainty is usually played down. In this case, though, there was very good DNA proof that the hair came from a bison. Had there not been this lingering element of mystery, I doubt the press would have been as interested as they were.

There was one other thing that Dr Coltman told me. The witnesses from Teslin were certainly well aware what a bison looks like, even from behind when only its two hind legs would be visible. They had recently shot one, their freezer was full of bison steaks and the pelt was hanging up in their cabin. Was the temptation to start a sasquatch rumour too great? If so, then it worked. Not many CNN news crews travel to the town of Teslin, Yukon during the course of a normal year.

The one exception to these rather casual liaisons between science and cryptozoology is the Sasquatch Genome Project run by Dr Melba Ketchum. Dr Ketchum is a qualified veterinarian who operates a DNA testing company in Timpson, Texas. The Sasquatch Genome Project website reports that the project has been under way for several years. Over the last couple there have been periodical releases of the project's progress, though nothing was formally published.

These releases were always eagerly anticipated by the cryptozoology community and when I met up with Bigfoot enthusiasts on my first tour of the US in early 2012, Dr Ketchum and the Sasquatch Genome Project was the one topic that dominated all others. Wherever I went, everyone I met wanted to know what I thought about it. I could only reply that until the research was published and I could have a good look at the data there was very little that I could say. That rarely put a stop to enquiries and was immediately followed by, ‘Well yes, but what do you think anyway?’ I did however admit that it is highly unusual for a professional scientist to release preliminary information about the outcome of a project intended for publication, except in confidence and for a good reason. Good quality science journals are reluctant to publish anything that is already in the public domain, and additionally, if there are intellectual property issues, disclosure will invalidate any patent application. So the early release of the Sasquatch Genome Project conclusions was highly unusual.

For example, the following explosive press release appeared in November 2012:

FOR IMMEDIATE RELEASE ‘BIGFOOT’ DNA
SEQUENCED IN UPCOMING GENETICS STUDY

Five-Year Genome Study Yields Evidence of Homo
sapiens/Unknown Hominin Hybrid Species in
North America

DALLAS, Nov. 24 – A team of scientists can verify that their 5-year long DNA study, currently under peer-review, confirms the existence of a novel hominin hybrid species, commonly called ‘Bigfoot’ or ‘Sasquatch,’ living in North America. Researchers' extensive DNA sequencing suggests that the legendary Sasquatch is a human relative that arose approximately 15,000 years ago as a hybrid cross of modern Homo sapiens with an unknown primate species.

The study was conducted by a team of experts in genetics, forensics, imaging and pathology, led by Dr Melba S. Ketchum of Nacogdoches, TX. In response to recent interest in the study, Dr Ketchum can confirm that her team has sequenced 3 complete Sasquatch nuclear genomes and determined the species is a human hybrid:

‘Our study has sequenced 20 whole mitochondrial genomes and utilized next generation sequencing to obtain 3 whole nuclear genomes from purported Sasquatch samples. The genome sequencing shows that Sasquatch mtDNA (mitochondrial DNA) is identical to modern Homo sapiens, but Sasquatch nuDNA (nuclear DNA) is a novel, unknown hominin related to Homo sapiens and other primate species. Our data indicate that the North American Sasquatch is a hybrid species, the result of males of an unknown hominin species crossing with female Homo sapiens.

‘Hominins are members of the taxonomic grouping Hominini, which includes all members of the genus Homo. Genetic testing has already ruled out Homo neanderthalensis and the Denisova hominin as contributors to Sasquatch mtDNA or nuDNA. The male progenitor that contributed the unknown sequence to this hybrid is unique as its DNA is more distantly removed from humans than other recently discovered hominins like the Denisovan individual,’ explains Ketchum in the press release.

‘Sasquatch nuclear DNA is incredibly novel and not at all what we had expected. While it has human nuclear DNA within its genome, there are also distinctly non-human, non-archaic hominin, and non-ape sequences. We describe it as a mosaic of human and novel non-human sequence. Further study is needed and is ongoing to better characterize and understand Sasquatch nuclear DNA.’

Ketchum is a veterinarian whose professional experience includes 27 years of research in genetics, including forensics. Early in her career she also practiced veterinary medicine, and she has previously been published as a participant in mapping the equine genome. She began testing the DNA of purported Sasquatch hair samples 5 years ago.

Ketchum calls on public officials and law enforcement to immediately recognise the Sasquatch as an indigenous people:

‘Genetically, the Sasquatch are a human hybrid with unambiguously modern human maternal ancestry. Government at all levels must recognise them as an indigenous people and immediately protect their human and Constitutional rights against those who would see in their physical and cultural differences a “license” to hunt, trap, or kill them.’

Full details of the study will be presented in the near future when the study manuscript publishes.

If, as is suggested by the last line, the study was about to be published then any journal would be furious with this premature release of the content and would probably pull the article. Journals are naturally keen to make the most of the publicity generated by original articles, especially one as spectacular as this one promised to be.

In February 2013, the promised article was published, but not in a peer-reviewed scientific journal. It appeared in a new web-based publication called ‘De Novo’ and was the only article in the first and so far only volume.

I have been asked on numerous occasions to comment on Dr Ketchum's work, so I decided the best way to do it was to write as if I had myself been asked to review the ‘De Novo’ manuscript for a scientific journal. This is something I have done on very many occasions and I soon found myself able to go into full reviewer mode. The following paragraphs are extracts from my review of the manuscript:

To the Editor.

Thank you for asking me to review this manuscript. I must first declare a conflict of interest in that results from my own laboratory completely disagree with those set out in this MS.

The manuscript describes a series of DNA tests performed on various tissue samples attributed to the North American sasquatch, a large, bipedal and perhaps mythical creature reported to be living in America, especially in the Pacific Northwest.

The main, and dramatic, conclusion is spelled out in the abstract: namely that ‘the data indicates (sic) that the Sasquatch has human mitochondrial DNA but possesses nuclear DNA that is a structural mosaic consisting of human and novel non-human DNA.’

Table 1 lists brief details of the 111 samples donated to the project. Three are saliva samples from either a food trap (#22,23) or a game camera (#27), one is mucus found on tree bark (#98), three are samples of tree bark (#47, 48, 49), three are dried blood samples (#25, 28, 107,110) and one is a toenail (#26). The other samples are hair, one with tissue attached (#18). This is a large sample and the authors are to be congratulated for assembling so much material.

There is a brief summary of the microscopic and electron microscopic evaluation of the hair samples which claims that most of the hairs can not be attributed to any known species held in their ‘reference collection’. This sweeping claim disregards the great difficulty in identifying the species origin of hair from microscopic appearance in most taxa, especially if the sample consists of one or a few hairs.

The authors state that the only hairs that were taken through to DNA analysis were those having visibly attached follicles. I am surprised so many of the donated samples passed this test as most were shed hairs that normally do not retain intact follicles.

The next stage was the extraction of DNA and sequencing of two regions of mitochondrial DNA, namely cytochrome b and hypervariable region I (HVS 1) of the control region. All 111 samples gave results at both cytochrome b and HVS 1 segments, which is an astonishing achievement. All samples revealed a human cytochrome b sequence and one of a range of different human sequences at HVS 1.

From these results and some unclear nuclear DNA data the authors conclude that the sasquatch is a hybrid between human females, which, in the authors' opinion, accounts for the presence of maternally inherited human mitochondrial DNA, and males of another unidentified species.

The real problem with this paper is that the authors have interpreted all the DNA results as supporting the hybrid genome hypothesis, while disregarding any alternatives. For example, the other parental species of the hypothesised hybrid is never identified and only assumed to be present at all because the nuclear DNA sequencing is not as one might expect from a good human sample. In my opinion the much more likely reason is that these are highly degraded and contaminated samples, despite the authors' efforts to argue otherwise. Such material is notoriously difficult to sequence and the authors have provided no convincing proof that their results are not entirely created by these well-known difficulties. In comparison, the efforts to by-pass human and environmental contamination in the sequencing of the Neanderthal genome (Green et al. 2010) took years of work to achieve.

The mitochondrial HVS 1 results show a wide range of sequence types whose continental origin can be deduced from their known geographical range. Of the twenty-five reported, nineteen are typically European, two are African and four are either Asian or Native American. To interpret these results as proof that the hybrids crossed the Bering land bridge from Siberia into North America is absurd. Despite the reassurances of the authors that they had eliminated all human contamination, it is far more likely that the majority if not all of the mitochondrial DNA sequences reported are the result of just that – human contamination.

This MS is not well written, and the project planning is poor. A lot of the material in this MS is unnecessary. It is a manuscript about DNA and any new version should restrict itself to exactly that. We do not need images of hair, trees, shelters, indistinct video clips and that sort of thing.

My advice to you is to reject this MS without offering the opportunity for revision with the same data. While the topic is certainly of great interest, the data do not support what is theoretically a most unlikely hybrid origin for Sasquatch.

My advice to the authors is to concentrate on the best sample and obtain a full genome sequence with at least 20x coverage. From that, and especially from any contiguous segments containing both genomic contributors, identify the other parental species, if indeed there is one.

Yours sincerely

Bryan Sykes

The Ketchum study never gets close to providing the exceptional proofs that such exceptional claims require. It may have begun as a promising project, but it was very poorly executed, has wasted a lot of valuable material, caused a great deal of confusion among cryptozoologists and must have cost someone a lot of money.

Caveat donor.