Reflecting on the quality of a methodologically pluralist evaluation of a large-scale Indigenous health research collaboration in Australia

Background: Indigenous communities worldwide lead calls for all evaluations of research, programmes and policies affecting their communities to reflect the values, priorities and perspectives of the Indigenous peoples and communities involved. Tools, such as the Quality Appraisal Tool (QAT), are available to assess research quality through an Indigenous cultural lens. Good evaluation requires that evaluation efforts be evaluated. We found that critical reflection on the quality of evaluations from an Indigenous perspective is largely absent from the published literature. To ensure that we strive for quality in evaluation as determined by Indigenous people with whom we work, we examined the quality of our own evaluation of an Indigenous health research collaboration by conducting a reflexive dialogue.

Methods: The QAT was used to assess our evaluation according to Indigenous health research principles. Our qualitative study used analytical coautoethnography to generate data through a series of reflexive dialogue sessions with Indigenous and non-Indigenous members of the research collaboration, using the QAT criteria as discussion prompts. Our ideas and reflections were compared and contrasted through a collaborative and iterative writing process, multiple review cycles and discussions.

Results: We documented our findings against the QAT framework. We found examples that each QAT principle had, to some extent, been adhered to, but constantly needed to assess whether the principles were fully achieved to our satisfaction. Strengths of the evaluation included being adaptable and responsive to emerging issues for the research collaboration, while areas for improvement included more Indigenous leadership of, and involvement in, evaluation.

Conclusions: Although reflexive evaluation practice is not always comfortable, it does provide an opportunity to generate insights for improvement. Reflecting as we did-in a partnership between Indigenous and non-Indigenous colleagues-enabled deeper insights and meaning. We anticipate that our process models how other research in Indigenous contexts might better advance ethical, quality Indigenous research through working in collaboration with Indigenous researchers and communities.