In the area of healthcare and health research surveys, it can be quite clear when participants or patients are providing inaccurate responses, but not always. In some cases, participants may be lying or actively trolling studies. They may have accidentally specified the wrong answer due to a survey that is badly designed for mobile technology or for an easy participant user experience. They may provide inconsistent answers due to the highly subjective nature of a question’s phrasing. Regardless, it’s safe to say that data can be compromised when participants provide inaccurate responses to surveys.
During our latest podcast Improving Surveys for Higher Completion Using Skillful Methodology, we sat down with our in-house Survey Methodologist Megan Ruxton, PhD to help us understand what makes a good survey. Here in this blog, we recap some examples she told us of why participants lie or provide inaccurate information. And we address some potential solutions that you and your research teams can use to mitigate bad data into your health research studies.
Watch the full podcast episode here:
In the era of digital health research—as in in all things digital, you may encounter bots or survey trolls that are purposely providing bad data or inaccurate answers to throw off the reliability of a study’s measures. Luckily, technology like two-factor authentication, CAPTCHA and other verification techniques can screen out trolls and bots so that data quality is not extensively affected. More recently, Ruxton explained that her colleagues in survey methodology are studying and incorporating machine learning to identify and mitigate fraudulent or otherwise inaccurate responses. And while there is not a solution to completely mitigate this possibility for research teams, digital health research platforms like Vibrent’s can vastly reduce the likelihood of trolls and bots with security best practices and the latest proven opt-in methods.
Ruxton goes on to say that there is a generational or cultural gap often where many older participants aren’t comfortable sharing certain health information they consider to be private. This could be information they may just not know, like family medical history, or something they’re simply too ashamed to admit. “It may be that you’re asking a generation that just doesn’t talk about sensitive topics,” Ruxton added. A good example may be health research surveys asking about sexual history or history of sexually transmitted infections, or an emotionally difficult health experience such as miscarriages.
The key in these situations is to be able to build trust with participants. You and your teams can find ways to preface these kinds of sensitive topics with guidance about the purpose of the question, trigger warnings, or to perhaps use a computer-assisted telephone interviewing (CATI) tool to conduct a live call with a person to give reassurance or support when navigating potentially upsetting questions around medical history. Surveys can also work up to more sensitive topics so that participants don’t feel blindsided with particularly intrusive questions.
There may be the issue that researchers are simply asking the wrong question. “Either someone doesn’t know the correct answer and is either not providing a response or giving you their best guess, or they’re interpreting the question in a way you don’t intend them to,” Ruxton explained.
Or the question’s wording could have multiple interpretations. “This is where it’s advantageous to have multiple eyes on a survey to see how people may be interpreting your questions. Maybe you’re asking something in a way where respondents think they’re giving you the right answer, but it’s not quite what you meant,” Ruxton said.
While bots and trolls are purposely lying, most respondents are not providing inaccurate data intentionally. When mitigating the risk of inaccurate data, it’s key to employ skillful methodologists who can help you word questions and structure health research surveys and questionnaires in a way that fosters trust and clarity for respondents. And in order to avoid the chance of some data trolls or bots, having a technology provider who can help you screen out these kinds of fake participants with the latest digital best practices helps to ensure higher data quality than working with outdated or ineffective polling platforms.