THE NEW SHIBBOLETH

'Say now Shibboleth'; and he said 'Sibboleth'; for he could not frame to pronounce it right
Book of Judges, 12:6

In the biblical story of the Shibboleth (literally, an ear of grain in Hebrew, and some say a rushing stream), a single mispronounced word separated insider from outsider. Since then, it has named any indicator that identifies someone as one of us or one of them, a human being or less of one. The Shibboleth has always defined what it means to be considered human.

Today, a new version of this test is playing out between humans and AI, as readers search AI-generated language for indicators that disclose its identity. Yet, research consistently shows that most users across fields fail to distinguish between human and generated texts. 

The New Shibboleth Project explores what happens in a world where users can no longer reliably tell whether a text was created by a human or a machine. The Turing test, which has long shaped how we think about this boundary, was fundamentally a test of performance: can a machine imitate a human well enough to fool a judge? The Shibboleth is a test of belonging: does the speaker display the signs that mark them as one of us? The question is no longer whether machines can pass a Turing test. Now we are asking what kinds of cultural tensions the constant search for the new Shibboleth creates and what it reveals about perceptions of what is distinctly human. 

The project has three purposes:

1. Mapping the critical thought and cultural representation of the Shibboleth across history, and providing a theoretical framework for understanding the Shibboleth in the case of nonhuman entities.

2. Work in progress: a timeline of the shifting indicators that users recognize as AI’s Shibboleths since large language models went public. Analyzing what they reveal about perceptions of what makes language human. 

This is a rare moment in the history of language. We intend to understand it.