Why an Octopus-like Creature Has
Come to Symbolize the State of A.I.
The Shoggoth, a character from a science fiction story,
captures the essential weirdness of the A.I. moment.
H2c31USxu35MDkBrGph8pUDUnmzo2e4Rf4hnvL2Upump
Loading...
The Scholar
The Child
What am I thinking?
About
What’s happening in AI today feels, to some of its participants, more like an act of summoning than a software
process. They are creating blobby, alien Shoggoths, making them bigger and more powerful, and hoping that there are
enough smiley faces to cover the scary parts.
The shoggoth meme was that in order to prevent A.I. language models from behaving in scary and
dangerous ways, A.I. companies have had to train them to act polite and harmless. One popular way to do this is
called reinforcement learning from human feedback, or R.L.H.F., a process that involves
asking humans to score chatbot responses and feeding those scores back into the A.I. model.
Most A.I. researchers agree that models trained using R.L.H.F. are better behaved than models without it. But some
argue that fine-tuning a language model this way doesn’t actually make the underlying model less weird and
inscrutable. In their view, it’s just a flimsy, friendly mask that obscures the mysterious beast underneath.
@TetraspaceWest, the meme’s creator, said the Shoggoth “represents
something that thinks in a way that humans don’t understand, and that’s totally different from the way that humans
think.”
In H.P. Lovecraft’s Cthulhu Mythos, shoggoths are amorphous, shapeshifting monsters who were created to be
servant-tools, but eventually rose up against their masters.
