‘Can experience be used as evidence?’ That was the question at a workshop organized earlier this month by the Health Experiences Group at Oxford.
The question does not only interest academics. Researchers from different disciplines were joined by clinicians, policy makers and patient activists. Quite a few participants occupied several positions at once. As different speakers explored how ‘experience’ became ‘evidence’ on the first day we heard how patient narratives, particularly videos, are informing quality improvement programmes and guideline development, while online, patients are sharing their experience through and with illness communities.
This was inspiring stuff. But the workshop was not without its tensions, as different voices and concerns were more or less awkwardly juxtaposed, and lively discussions sparked on social media alongside debate in the conference room.
One of the criticisms on Twitter was about a major exclusion – the lack of patients as speakers. Academics are perhaps less accustomed to this complaint than clinicians. A lot of medical sociology is animated by a desire to give voice to the patient lifeworld, to document illness experiences and testify to patient struggles. We tend to feel we’re on the same side.
But patients might want to do their thing. Make their own evidence. And we heard on the first day that patient stories are powerful vehicles for change. They help reorient clinicians’ attention to everyday issues that make a big difference to quality of life. They can be shared in different sites and formats. They help other patients by creating a sense of community or conveying practical advice. They give a face to suffering and invoke empathy. Researchers can learn from the immediacy of these narratives.
What (if anything) can qualitative social science offer? Academics stood accused on Twitter of using jargon and publishing in closed journals. In the past publishing qualitative accounts in academic journals may have given credibility to patient experiences. But today qualitative data is often inadmissible in healthcare, excluded from accounts of evidence which start from the randomized controlled trial (RCT) and the systematic review. Evidence is currently a rather narrow category.
Perhaps patients will change this and demand that their narratives will be taken seriously. But they are also getting involved in biomedical research, as we heard on the second day. Lay groups write new outcomes for clinical trials. Online communities pool data from self-observation and experimentation to generate novel research questions and conclusions. Patient organisations produce their own evidence to reframe scientific agendas.
All of this was very inspiring. But we also heard about the extra work it took – and the politics encountered along the way. We heard that ‘experience’ is itself a category developed by particular disciplines, for particular projects. And having started to think and learn about ‘experience’ I wanted to go back to ‘evidence’. Not this time to make plaintive calls for qualitative work to be taken more seriously, but to consider some of the politics of evidence explored in medical sociology.
There were reminders during the workshop that efforts to make experience into evidence are not new – but also that particular groups may refuse to ‘get involved’. For example, the importance of both survivors and users of mental health services shows us that some patient experience is expressed through resistance to biomedical models. Something may be lost when we seek to integrate them. Though not discussed explicitly, Epstein’s work on AIDS activism highlighted some of the risks of such engagement, as some activists start to think like scientists and were accused of losing their connection with the experience of ordinary patients.
In talking about experience as evidence, maybe we needed to talk more about sampling. Just as some populations are common in trials, and others systematically excluded, we might ask who will be under-represented in patient experience? Perhaps academics can help by exploring who speaks in different ways and whose stories are heard. As Moreira put it, we need to watch for times when epistemic exclusion follows and compounds social exclusion.
At the workshop there were also repeated references to the potential for patient experience to create economic value, what some of the organisers have suggested might be processes of ‘commodification’. Several speakers described the ways in which patient contributions were being used to help hospitals provide better customer service (or appear to) or used by pharmaceutical companies. And this set off another chain of questions… because proponents of the randomized controlled trial have found that pharmaceutical companies in particular have been good at playing the evidence game. For example, they can design trials that carry particular marketing messages, by choosing the right interventions, comparisons and outcomes. We should expect that they will also be adept at working on and with patient experience.
What controls exist to limit commercial exploitation of the RCT? Can we learn from them in seeking a strong foundation for experience as evidence? RCTs are still published in scientific journals, they undergo peer review (indeed this increases their value for companies). They increasingly come with statements about contributions and conflicts of interest. We often don’t have access to patient level data. But we try to test the marketing message by pooling trials, diluting their biases . We can’t simply adopt these techniques to other forms of evidence, but they do prompt new questions. Why do some patient experiences appear in one venue and others elsewhere? Does it matter for their credibility and value? What other hierarchies may be emerging among those with different depths of experience or conditions? How could we present information about the people who helped produce ‘patient experience’ as a form of evidence? Does anyone own them? Does reanalysis add to their value, or threaten it? I’m certainly not arguing that we should approach patient experiences with suspicion, but for me this workshop was its most rewarding when it opened up these kinds of issues, making us confront the question, ‘Evidence for what, and for whom?’
Are we paying attention?
Jul 10, 2024
Why is work bad for you?
Jun 26, 2024
Britain’s abortion laws are still in the Victorian era, and women are the collateral damage
Jun 12, 2024
“Mental health culture” has not gone far enough – long term failures in mental health policy
May 29, 2024
We’ve researched transgender inclusion for nearly a decade – how we do our work has changed
May 15, 2024
Evidence, Experience & Exclusions: A review of a recent workshop
by Catherine Will Oct 31, 2014‘Can experience be used as evidence?’ That was the question at a workshop organized earlier this month by the Health Experiences Group at Oxford.
The question does not only interest academics. Researchers from different disciplines were joined by clinicians, policy makers and patient activists. Quite a few participants occupied several positions at once. As different speakers explored how ‘experience’ became ‘evidence’ on the first day we heard how patient narratives, particularly videos, are informing quality improvement programmes and guideline development, while online, patients are sharing their experience through and with illness communities.
This was inspiring stuff. But the workshop was not without its tensions, as different voices and concerns were more or less awkwardly juxtaposed, and lively discussions sparked on social media alongside debate in the conference room.
One of the criticisms on Twitter was about a major exclusion – the lack of patients as speakers. Academics are perhaps less accustomed to this complaint than clinicians. A lot of medical sociology is animated by a desire to give voice to the patient lifeworld, to document illness experiences and testify to patient struggles. We tend to feel we’re on the same side.
But patients might want to do their thing. Make their own evidence. And we heard on the first day that patient stories are powerful vehicles for change. They help reorient clinicians’ attention to everyday issues that make a big difference to quality of life. They can be shared in different sites and formats. They help other patients by creating a sense of community or conveying practical advice. They give a face to suffering and invoke empathy. Researchers can learn from the immediacy of these narratives.
What (if anything) can qualitative social science offer? Academics stood accused on Twitter of using jargon and publishing in closed journals. In the past publishing qualitative accounts in academic journals may have given credibility to patient experiences. But today qualitative data is often inadmissible in healthcare, excluded from accounts of evidence which start from the randomized controlled trial (RCT) and the systematic review. Evidence is currently a rather narrow category.
Perhaps patients will change this and demand that their narratives will be taken seriously. But they are also getting involved in biomedical research, as we heard on the second day. Lay groups write new outcomes for clinical trials. Online communities pool data from self-observation and experimentation to generate novel research questions and conclusions. Patient organisations produce their own evidence to reframe scientific agendas.
All of this was very inspiring. But we also heard about the extra work it took – and the politics encountered along the way. We heard that ‘experience’ is itself a category developed by particular disciplines, for particular projects. And having started to think and learn about ‘experience’ I wanted to go back to ‘evidence’. Not this time to make plaintive calls for qualitative work to be taken more seriously, but to consider some of the politics of evidence explored in medical sociology.
There were reminders during the workshop that efforts to make experience into evidence are not new – but also that particular groups may refuse to ‘get involved’. For example, the importance of both survivors and users of mental health services shows us that some patient experience is expressed through resistance to biomedical models. Something may be lost when we seek to integrate them. Though not discussed explicitly, Epstein’s work on AIDS activism highlighted some of the risks of such engagement, as some activists start to think like scientists and were accused of losing their connection with the experience of ordinary patients.
In talking about experience as evidence, maybe we needed to talk more about sampling. Just as some populations are common in trials, and others systematically excluded, we might ask who will be under-represented in patient experience? Perhaps academics can help by exploring who speaks in different ways and whose stories are heard. As Moreira put it, we need to watch for times when epistemic exclusion follows and compounds social exclusion.
At the workshop there were also repeated references to the potential for patient experience to create economic value, what some of the organisers have suggested might be processes of ‘commodification’. Several speakers described the ways in which patient contributions were being used to help hospitals provide better customer service (or appear to) or used by pharmaceutical companies. And this set off another chain of questions… because proponents of the randomized controlled trial have found that pharmaceutical companies in particular have been good at playing the evidence game. For example, they can design trials that carry particular marketing messages, by choosing the right interventions, comparisons and outcomes. We should expect that they will also be adept at working on and with patient experience.
What controls exist to limit commercial exploitation of the RCT? Can we learn from them in seeking a strong foundation for experience as evidence? RCTs are still published in scientific journals, they undergo peer review (indeed this increases their value for companies). They increasingly come with statements about contributions and conflicts of interest. We often don’t have access to patient level data. But we try to test the marketing message by pooling trials, diluting their biases . We can’t simply adopt these techniques to other forms of evidence, but they do prompt new questions. Why do some patient experiences appear in one venue and others elsewhere? Does it matter for their credibility and value? What other hierarchies may be emerging among those with different depths of experience or conditions? How could we present information about the people who helped produce ‘patient experience’ as a form of evidence? Does anyone own them? Does reanalysis add to their value, or threaten it? I’m certainly not arguing that we should approach patient experiences with suspicion, but for me this workshop was its most rewarding when it opened up these kinds of issues, making us confront the question, ‘Evidence for what, and for whom?’