Artificial intelligence has a pretty poor track record of trying to fool humans. From attempting to write Game of Thrones books to creating Dungeons and Dragons monsters, it just doesn’t quite have the creative knack of humans.
But then, it looks like experts aren’t that good at telling the difference between human and machine either, at least when it comes to poetry. A study published on arXiv from Kyoto University found that, despite writing some truly terribly verses, experts couldn’t tell if it was written by AI, as Futurism pointed out.
How bad was the poetry? Well, we’ll let you be the judge. Here’s one of the poems it wrote.
the sun is a beautiful thing
in silence is drawn
between the trees
only the beginning of light
And here’s another, which actually is pretty great.
I am a coal-truck
by a broken heart
I have no sound
the sound of my heart
I am not
To create the poems they trained a neural network to analyze thousands of human-made poems, and then have a go at creating its own. Humans were then also asked to submit some poems, to see how they compared to the AI.
And you might be surprised to learn that in the study, they actually fooled people. When comparing AI poems to human poems, a group of more than 500 people – including 30 experts – got it wrong more often than not.
“The generated poems have caused a competitive confusion to both ordinary annotators and experts, though experts can figure out the accurate one better than ordinary people,” the researchers wrote in their paper.
People took part in the study through Amazon’s Mechanical Turk, an online survey tool. Some of the poems accompanied images, which were supposed to be related, and others didn’t.
The researchers claimed this was a “Turing Test” of sorts, which is a test to see if AI machines can fool people into thinking they are human. While it’s probably a stretch to say that’s the case, it does highlight that AI isn’t the creative dud we thought it was, at least when it comes to poetry.
Or, you know, people are just really bad at analyzing poetry. Take your pick. Here’s a couple more of the AI poems, just to brighten your day.
the sun is shining
the wind moves
naked trees
you dance
And one more.
and now I am tired of my own
let me be the freshening blue
haunted through the sky bare and cold water
warm blue air shimmering
brightly never arrives
it seems to say
1
