Reproducibility is one of the core features of science, so experiments should produce the same results regardless of who conducts them. However, disturbing evidence that some studies have different outcomes depending on whether they were done by men or women has come to light. A Science Advances paper summarizes the evidence, and discusses how the phenomenon should be addressed.
The idea that an experimenter's gender could affect the outcome of their research might sound pseudo-scientific, at least if you are thinking of studies in astronomy, chemistry, or geology. If these fields are shaped this way then we've got bigger problems than we know. However, evidence of unexpected effects has emerged in some social sciences.
For example, PhD student Colin Chapman and Professor Helgi Schiöth of Uppsala University, Sweden, note pain studies have been found to be distorted because men will report less pain when the experiment is being run by a woman instead of a man. It's not clear whether men are, on average, more reluctant to admit suffering to a woman, or are soothed by her presence. Either way, trials of pain relief drugs against placebos don't work if one was administered by a man and the other by a woman.
On consideration, the pain example is not so surprising, but Chapman and Schiöth note there are other examples that may have more widespread influence. When intelligence tests are given by women, measures of verbal intelligence, comprehension, and vocabulary scores come out higher on average than if given by men. Interestingly this effect has been measured on children, both boys and girls, rather than adults where sexual attraction might drive attempts to impress the experimenter.
Even mice are more stressed by male lab assistants.
On the other hand, male experimenters get more responses to creative problem-solving tasks, mainly from women. Studies in areas such as aggression and physical performance have produced inconsistent results.
As Chapman and Schiöth note, this is more than an intriguing quirk if reliance is placed on studies where there was no consistency in the gender of the experimenter. “[Particularly] alarming is the impact on the development of therapeutics to treat learning disabilities in children,” they write. The same applies to medications for ADHD.
These patterns might help explain psychology's “reproducibility crisis”, where many apparently well-conducted studies cannot be successfully replicated.
Ideally, the authors propose, many studies should be consistent in the gender of the experimenter, but at the very least the gender of those conducting the experiments should be reported, something that seldom happens now.
Gender is just the beginning. In some cases, the age, personality, and even height of experimenters have been proven to be influential, but these effects are more difficult to track since they more often occur on a graduated scale.