Technology
PUBLISHED
A farmer in China has claimed he lost around 100,000 square meters of crop overnight after following advice given to him by an artificial intelligence (AI) chatbot.
AI chatbots, as they will happily explain to you, are not truth machines. They are trained on large datasets of (largely) human-produced input, they learn the patterns of how words and sentences fit together, and then they spit out words in the order the algorithm predicts will be most pleasing to the user it is "conversing" with.
Sometimes that overlaps with the truth, but not always. A recent international study, for example, found that 45 percent of answers given by AI had at least one significant issue with them, with 20 percent of them containing "hallucinations" or "bullshit" as it is known when humans start spouting similar made-up nonsense.
That's a little concerning, with some research showing that one in seven people have used AI for health advice, and other research showing that users who are vulnerable, less proficient in English, or less-well educated are often served less accurate information than others.
A lot of time that might not have too many big consequences. If you're asking "how many rocks should I eat?" as a joke, for example, you are probably still sensible enough to know not to trust it when it recommends "at least one." But occasionally, given how many people are exposed to AI through choice or through the insistence of tech companies like Google, these answers can have negative, real-world consequences.
Back to farming. According to sources in mainland China, per Taiwanese news outlet CTWant, a 67-year-old farmer in Chuzhou, China, became reliant on AI advice over the course of several months. The man reportedly used an unnamed chatbot to organize farming schedules, as well as garner advice on fertilizers. Though skeptical of the advice at first, he came to trust the answers it was spitting out at him.
In July, he reportedly paid a hefty price for his trust in the chatbot after he asked for advice on how to control pests on his sesame crop. Per International Business Times, the chatbot recommended flupyrimethalin and fomesafen, mixed with thiamethoxazine and methyl salt, which the farmer then sprayed on his sesame crops without seeking human advice.
The next day, the farmer reportedly found that around 10 hectares (100,000 square meters) of his crops had been destroyed.
Following this, the man turned to the chatbot once more to figure out what the problem might be. This time it was more helpful, identifying that fomesafen is a herbicide largely used to control broadleaf weeds, often in soya crops. As sesame is a broadleaf species, the chatbot informed the farmer (again with no authority on truth) it was unsuitable to use a herbicide which targets broadleaf species.
"If you spray it, the next day the seedlings won't survive," the farmer told CTWant. "Both the grass and the seedlings will die, and the seedlings will die even faster."
This is far from the first AI error that has had real-world consequences, and it won't be the last either. For an example that made it to peer review, one case report documents a man who was recommended to replace sodium chloride with sodium bromide, with predictably disastrous results.
