Dear George,
Can you tell if a poem has been written by a human versus by an AI chatbot? Below are ten pairs of limericks. Within each pair one limerick was written by a human being (myself) and the other was written by Gemini, Google’s artificial intelligence chatbot. Gemini was instructed to write a limerick on the same topic as my human-written version (e.g., “Write a limerick about bugs in the kitchen”).
Your task is to judge which is which. First write the numbers 1 through 10 down a blank piece of paper. Then read each pair of limericks. If you think the first poem in the pair was human-written and the second was AI-written, write H/AI next to the appropriate number. Conversely, if you think the first was AI-written and the second was human-written, write AI/H next to the number. As a hint, half of the pairs are in one order and half are in the other order. Correct answers will be given at the end.
1. BUGS IN THE KITCHEN
The small insects, countless and bold
In our kitchen their story unrolled
They marched 'cross the floor
Then asked us for more
A sticky sweet chaos of old
Hordes of bugs have invaded our sink
Seeking morsels to eat and to drink
Some are big, most are small
All delighted to crawl
These bugs are night creatures, I think.
2. AGE EIGHTY-FIVE
I’ve hit eighty-five years on the spot
And still feel like a colt that can trot
My memory's a blur
But of this I am sure,
I've learned what to do and what not
Today I turned age eighty-five
Quite puzzling to still be alive
Never planned for this day
Crummy hearing, hair gray
The new question, how best to survive
3. WORDLE IN TWO GUESSES
Today I solved Wordle in two
Success leaped straight out of the blue
My first word was CREST
Got three letters, guessed the rest
Offbeat picks but they proved to be true
To start, I chose ADIEU with great thought
For the second guess, the right word was caught
In guesses just two
The answer broke through
A Wordle solved quickly, well wrought
4. BENGALS LOSE TO BEARS
The Bengals' defense, it is said
Left the Bears' tight end quite well fed
A big lead erased
Their victory displaced
Now Cincy fans just shake their head
The Bengals got thumped by the Bears
Though our offense gained more yards than theirs
Near the end, things looked good
Victory looming, knock on wood
Oh no, Zack! Please make some repairs
5. OUR DOGGIE’S ACCIDENT
Iko pooped right on top of our bed
He must have been out of his head
I took him straight out
Screamed that he was a lout
And now we just eye him with dread
Young Iko, a pup full of zest
Put his owners quite fully to the test
On the bed, soft and deep,
While his people did sleep
He left a brown gift, unimpressed
6. LEFTOVER THANKSGIVING TURKEY
We are still eating leftover turkey
Mixed with chicken a la king, sort of murky
Most years turkey’s my favorite
I've always said I savor it
But right now I would rather have beef jerky
I thought I would finish the bird
Every last morsel, uncured
My stomach then cried
"You're quite over-plied!"
Now I'm stuck, very groggy and blurred
7. GET A HAIRCUT
My dear wife, she looked quite aghast,
"That mop top is growing too fast!
Like a shaggy old mutt,
You look like a nut,
Get a trim, before winter has passed!”
My wife tells me, “Get a haircut”
She claims people think I’m a nut
But I don’t want to go
My hair’s still so-so
And who cares if I look like a mutt
8. TO THE BODY SHOP
I’ve bashed in the blue Honda’s fender
It looks like I’ve been on a bender
Backed up into a wall
Now I’m off to John Hall
That John Hall's quite skilled, he will mend ‘er
My blue Honda suffered a smash,
Its front fender looked like pure trash.
To John Hall's I drove slow,
Where they put on a show,
To fix up the dent in a flash!
9. DINNER AT THE CHART HOUSE
The Chart House for my birthday dinner
Holy moly, I felt like a winner
I ordered sea bass
Such a fish, it’s first class
But the lava cake won’t make me thinner
For my birthday, I went to the spot
The Chart House, all fancy and hot
I savored the bass
Then ate with great class,
The warm, gooey lava cake pot
10. THE WINTER SNOW STORM
A huge storm brought snowfall so deep
Eight cold inches started to creep
"Dear Iko, I fear
Will you stay warm, my dear?"
While he snored in a comforting heap
We’re enjoying a huge winter storm
The whitest of white is the norm
The snow’s eight inches deep
Our car’s under the heap
I just hope that our pup Iko can stay warm.
How did you do? You might want to guess how many of the 10 pairs you got correct before reading the following answers. Here is the answer key. Give yourself one point for each pair that you got correct. KEY: 1. AI/H; 2. AI/H; 3. H/AI; 4. AI/H; 5. H/AI; 6. H/AI; 7. AI/H; 8. H/AI; 9. H/AI; 10. AI/H.
I bet you did better than you thought. I tried this quiz out on members of my writers’ group, and they thought it was pretty hard. However, most got between six and nine pairs correct. (Because it’s a 50/50 decision, a score of 5 would be chance.). When I imagined doing the quiz, I thought odd rhymes were a cue to AI-generated poems (e.g., “unrolled” in pair 1; “wrought” in pair 3; “unimpressed” in pair 5; "pot" in pair 9). On the other hand, I thought some pairs were almost impossible to distinguish (e.g., 2, 7).
There has been research on whether people can reliably distinguish between poems written by classic authors (e.g., Shakespeare, Emily Dickinson, Walt Whitman, T.S Eliot) and AI-generated poems (when the chatbot was instructed to write in the style of a given author). There are two main findings. (1) Participants were generally unable to distinguish the two classes of poems, sometimes more likely to judge an AI-generated poem as human-authored than a poem actually written by a human author. (2) When not told whether an author was human or AI, participants often rated AI-generated poems more favorably. The researchers have suggested that non-expert readers prefer AI-generated poems because they are generally more straightforward, grammatically correct, and easier to understand versus the complexity and ambiguity of poems written by classic human poets.
So the big question is whether poetry writing is going to be taken over by Artificial Intelligence. I don’t think it will, though AI is getting more sophisticated, year by year. Probably the best answer was given by one of my fellow group members: “I wouldn’t have any interest in reading a poem written by a chatbot.”
Love,
Dave
