Going viral on the Internet can be a blessing or a curse, and occasionally both. That’s exactly what happened to the scientific survey platform Prolific. A teenager's TikTok video spread their name to millions, usually good when looking for people to take part in a survey, but then they had to deal with some unexpected consequences of their viral fame: the disruption to thousands of scientific investigations.
Let’s rewind the clock to July 23. Florida high school graduate Sarah Franklin posted a video on TikTok talking about how you can get paid to take surveys on sites like Prolific.co.
“Welcome to side hustles I recommend trying — part one,” she says in the video. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.”
The video was shared widely and so many people got involved that Prolific posted on Reddit about their viral fame. However, pride comes before a fall, as the English proverb goes. The team at Prolific had not considered that even though their numbers were up, the audience of a teenage TikTokker might not be representative of the US population in general, something vital for surveys that rely on a wide mix of subjects for responses.
Data from the company show that before July 24, participants were more or less equal people assigned male at birth and people assigned female at birth (AMAB and AFAB, respectively) and range of ages. In the days after the video went viral, perplexed scientists discovered that 90 percent of the respondents to their survey were female with an average age of 21.
And while there are existing major gendered biases towards men when it comes to science, and more women should be included in research and trials, these won’t be fixed by having such skewed samples as this – especially since the surveyors were expecting participants to be representative of the general population.
The disruption, luckily, is not permanent. First of all, the data skew post-TikTok appears to be waning slowly, although the percentage of AFAB participants remains more than 60 percent, just not the three-quarters level that it reached in early August. According to The Verge, 4,600 studies were disrupted by the sudden influx of new participants.
Prolific has offered several fixes, and even compensation in some cases, to the survey makers, some of which have found ways to work around the data, analyzing the gender separately. Prolific has also allowed researchers to select respondents by "Participant Join Date" and use their representative sample screener for free.
“We're dedicated to empowering you to conduct great research on Prolific, and data quality is our top priority,” Prolific support lead Nick Charalambides said in a blog post.
"As well as the above measures, we'll also be more closely monitoring the pool of participants on Prolific to avoid similar spikes in signups, until we have stronger controls in place for letting you easily balance your studies across certain demographics.”
[H/T: The Verge]