Evolution News and Views (ENV) provides original reporting and analysis about the debate over intelligent design and evolution, including breaking news about scientific research.

Evolution News and Views
Intelligent Design NEWS

Some Scientists Say Intelligent Design Isn't Science -- Until They Have to Use It Themselves


Many scientists claim intelligent design is not science, until they have to use it. Then they find it very helpful. This is a story about how, out of necessity, science journal editors have had to use design principles to fight fraud. It's also illustrates the fact that ID makes no claims about the morality of a good design.

Back in 1996, Alan Sokal, a physicist, created a firestorm in academia by submitting a realistic-sounding but nonsensical paper to the postmodern journal Social Text. It was a pure hoax, intended to shame the followers of Derrida, Foucault, and other postmodern philosophers who were undermining the credibility of "scientific realists" by claiming that scientific theories are products of social forces, not the real world. When Sokal went public with his strategy, the publishers of the journal were understandably angry and embarrassed. How were they supposed to understand the arguments of a physicist? The incident raises another issue. Journals are supposed to be icons of integrity, but the motives of a submitter can no longer be assumed.

Lately, there's been a flood of fake papers submitted to scientific journals. Software can now generate papers that look and sound very realistic. How do the editors and peer reviewers separate the wheat from the chaff? They have had to use their own design strategies to develop cheat-detecting software. This is described by John Bohanon in Science Magazine, "Hoax-detecting software spots fake papers." The opening reads like a hacker nightmare:

It all started as a prank in 2005. Three computer science PhD students at the Massachusetts Institute of Technology -- Jeremy Stribling, Max Krohn, and Dan Aguayo -- created a program to generate nonsensical computer science research papers. The goal, says Stribling, now a software engineer in Palo Alto, California, was "to expose the lack of peer review at low-quality conferences that essentially scam researchers with publication and conference fees."

The program -- dubbed SCIgen -- soon found users across the globe, and before long its automatically generated creations were being accepted by scientific conferences and published in purportedly peer-reviewed journals ....

The bad publicity for publishers mounted in 2013, when 85 SCIgen papers were found in the published proceedings of 24 different computer science conferences between 2008 and 2011. More were soon discovered, and 122 nonsense conference papers were ultimately retracted by Springer, the academic publishing giant based in Heidelberg, Germany, and by the Institute of Electrical and Electronic Engineers, based in New York City. [Emphasis added.]

Unlike the Sokal affair, there's not one person responsible. This is more like a spambot attack. The ability of hackers to download and use prebuilt hacker tools has hit academia. How do you fight fire? With fire:

But SCIgen may have finally met its match. Last week, academic publisher Springer released SciDetect, a freely available program to automatically detect automatically generated papers.

Bohanon tells about how this has become a familiar arms race scenario: as the white hats get better, the black hats rise to the occasion. Like Sokal, the authors of SCIgen had an agenda: to shame the sloppy peer review at conferences. The readily available nonsense generator, though, now enables Chinese academics and students to pad their publication records. So much for integrity. For publishers, it's a "public-relations nightmare."

How does one go about detecting nefarious design? This is interesting from ID perspective, because the fake papers represent cases of false positives. ID theory allows for false positives (items that don't look designed but really are), but is robust against false negatives (items that are not designed but inferred to be designed). If the improbability of an item originating by chance or unguided processes exceeds a universal probability bound, a design inference is warranted.

SciDetect looks for patterns to discriminate and distinguish the true from the false. Cyril Labb�, a computer scientist at Joseph Fourier University in Grenoble, France, was hired by Springer to be the new sheriff in town.

In the wake of that public relations nightmare, Springer approached Labb� for help. He agreed, for a price -- enough to fund a 3-year PhD student, Springer says. Labb�'s method for finding the nonsense papers was sophisticated, requiring a statistical technique similar to spam e-mail detection, but based on grammatical patterns rather than on keywords like "Viagra."

We might note that SciGen was intelligently designed for a function, even if its output is intentional nonsense. The fake papers are designed to confuse an opponent, something like a barrage of dummy missiles attempting to thwart an automated defense system. Everything is designed in this predicament; what we see here is a signal-to-noise problem where the noise is intentional, a classic wheat-and-tares sorting problem.

ID handles these situations, too, because by definition, ID looks for "certain features" of a phenomenon that are "best explained" by an intelligent cause, rather than an "undirected cause." Even though SciGen is designed, and outputs English words and sentences, there's an undirected process in the way the output is generated, probably by means of a random-number generator in the algorithm. The ID definition, therefore, fits the situation.

The result is SciDetect, a program to automatically detect papers created with SCIgen and similar programs. Its purpose, according to Springer, is to "ensure that unfair methods and quick cheats do not go unnoticed."

So far, SciGen has dramatically reduced the number of gibberish papers at Springer conferences. One critic, however, worries that while it can detect automatically generated nonsense, it is not smart enough to catch a Sokal-type hoax. Springer replied, "Software cannot replace peer reviews and academic evaluation, but SciDetect lends publishers an additional hand in the fight against fraud and fake papers."

Bohanon gets into charges and counter-charges about the motives of the perpetrators and the publishers, but those ethical issues are beyond the scope of ID theory. ID can only detect design; it makes no claims about the identity or motives of the designer(s).

As for the pranksters, they will just have to work harder, says Stribling, the SCIgen creator. "I'm willing to bet if someone wanted to declare an arms race, they could come up with another way to generate papers that would fool [SciDetect] again for a while."

Yes, it's gratifying to see scientists relying on an idea -- intelligent design -- that they so often repudiate.

Image: Real apple and fake apple, by Arbitrarily0 (Own work) [CC BY-SA 3.0 or GFDL], via Wikimedia Commons.