Here, we reflect on the conceptualization, writing and publication of Advancing the Methodology, and describe three lessons drawn from the experience that have influenced our career trajectories.
We wrote Advancing the Methodology as trainees (DL and HC were PhD students, and KKO was a postdoctoral fellow) in the School of Rehabilitation Science at McMaster University. Each of us had recently used Arksey & O'Malley's (2005; fondly referred to hereafter as A&O) pioneering methodological paper to undertake an (at the time!) novel method of knowledge synthesis: the scoping review. Their paper offered an excellent road map, but also illuminated new challenges. A&O deftly recognized the work to be done, stating:
“One of the purposes of the present paper is to stimulate discussion about the merits of scoping studies, and help develop appropriate methods for conducting such reviews……We look forward to seeing how the debate progresses.”(p31)
As trainees, we felt ourselves to be less than ideal candidates to answer this call, but a fortuitous occasion of sage mentorship from DL’s doctoral supervisor, Dr. Cheryl Missiuna, changed our minds. Dr. Missiuna suggested that our collective experiences (published by DL in Research in Developmental Disabilities, HC in the Canandian Journal of Occupational Therapy , and KKO in Aids and Behavior) had value, and that the three of us partner to take on this challenge. Her encouragement gave us the confidence to undertake the work. She taught us Lesson #1: Look for ways to foster confidence in your trainees. She embodied this lesson by promoting our independence. Once the seed was planted to write a paper about our collective experiences, we were left to organize ourselves and do the work. Which we did! In addition to encouragement, Dr. Missiuna provided the funds to ensure the paper was open access. Her belief in our abilities, combined with the post-publication evidence of interest in the paper, nurtured our confidence as early career researchers, helping to establish us early on as methodologists in a rapidly developing field.
It helped that we were in the right place at the right time. Figure 1 illustrates the growth in scoping review (or scoping study) publications between 1997 and 2022, showing how the rate of scoping review publication was on the cusp of rapid acceleration following the 2005 A&O paper. However, we combined good luck with adequate preparation: we had each read widely, illuminating the gap that needed to be filled. In retrospect, we see this as Lesson #2: Question the status quo. We embody this lesson in nudging our students (and our colleagues!) to consider how advancing existing methodologies could be a component of their work.
For example, HC was an external examiner on a student’s comprehensive exam that occurred just a week before publication of the PRISMA scoping review reporting guidance. She asked the student to reflect on how the new guidance might influence her (already written in preparation for publication!) review. That question, along with sage encouragement from HC, spurred the student to write an editorial based on their experiences of altering their paper to reflect the new reporting guidance. We encourage our trainees to follow existing methodical guidance, but we also prompt them to continually reflect on how a methodology could be adapted to better address the research question. Over the years, lesson #2 has motivated us to question the other aspects of our field that could benefit from disrupting the status quo.
Importantly, we also benefited from mentors who believed in the value of collaboration. Combining our experiences into one paper made it stronger than if any one of us had written it alone, leading us to Lesson #3: Consider the process of doing the work to be equally important as its outcome. This can be difficult to remember in academia, where outcomes are key to advancement, specifically those that can be tallied in an annual progress or impact report. Yet the impactful, collaborative process of co-ideating and co-writing gave us an early model to replicate and expand in our subsequent careers.
Indeed, Advancing the Methodology spurred new collaborations with researchers doing similar knowledge synthesis methodological work, formalized in a Canadian Institutes of Health Research Knowledge Dissemination grant in 2015 led by KKO. The grant brought together an international working group in Toronto to discuss scoping review methodological improvements. At this meeting, Dr. O’Malley herself presented the historical perspectives and origins of the scoping study (NB: A&O used the term scoping study). Figure 2 shows one of our brainstorming boards from this meeting – as you can see, a rich idea generation experience!
We joined forces with other groups working in this same area to continue collaborative efforts, including Drs S. Strauss and A. Tricco's group at the University of Toronto, who had expertise in knowledge syntheses and developing reporting guidelines (including a scoping review published in 2016 in BMC Research Methodology. This collaboration led to a 2014 Editorial in the Journal of Clinical Epidemiology and the publication of the PRISMA guidelines for Scoping Reviews. Further collaborations expanded to include another team working on reporting guidance for scoping reviews: Mikah Peters from the Joanna Briggs Institute (JBI).
Amazingly, as of March 2022, Advancing the Methodology has been cited 6792 times (Google Scholar) and downloaded over 170,000 times. We never anticipated the attention the paper has received, nor the wide-ranging impacts of its publication on our careers. These impacts include the considerable time and energy spent on mentoring, advising, and presenting on scoping reviews, exposing us to a plethora of new research interests, perspectives, and worldviews.
Scoping reviews: Looking to the future
Many lively discussions have been held with the goal of debating the identity, conceptualization, definitions, and terminology related to scoping reviews. The most interesting were debates related to the value of quality assessment of included studies in a scoping review, the debate on terminology – is it a study, or is it a review? - and the pros and cons, from a scientific perspective, of the inherently iterative nature of the scoping process, as emphasized in the A&O paper. Interestingly, these issues remain relevant to this day. We also note emerging areas of debate such as how best to present scoping results for a wide range of stakeholders, how best to conduct a qualitative synthesis of data, and when to conduct the (optional) consultation exercise. We look forward to ongoing discussions!
We end with an expression of our collective gratitude for this experience. Publishing this paper had both measurable (the citations!) and immeasurable impacts on our career trajectories, influencing our dedication to the essential scientific pursuits of collaboration and mentorship. We would like to amplify A&O’s original call by encouraging others - at any stage of their careers - to join in methodological advancement efforts to benefit evidence-informed practice, research and policy.
The authors acknowledge the contributions of Sarah Zarshenas, Research Associate at University of Toronto, in gathering the data for and creating Figure 1. Kelly K. O’Brien (KKO) is supported by a Canada Research Chair in Episodic Disability and Rehabilitation from the Canada Research Chairs Program.
- Levac, D., Colquhoun, H. & O’Brien, K. K. Scoping studies: Advancing the methodology. Implement. Sci. 5, (2010).
- Arksey, H. & O’Malley, L. Scoping studies: towards a methodological framework. Int. J. Soc. Res. Methodol. 8, 19–32 (2005).
- Levac, D., Rivard, L. & Missiuna, C. Defining the active ingredients of interactive computer play interventions for children with neuromotor impairments: A scoping review. Res. Dev. Disabil. 33, (2012).
- Colquhoun, H. L., Letts, L. J., Law, M. C., MacDermid, J. C. & Missiuna, C. A. A scoping review of the use of theory in studies of knowledge translation. Can. J. Occup. Ther. 77, 270–279 (2010).
- O’Brien, K., Wilkins, A., Zack, E. & Solomon, P. Scoping the field: identifying key research priorities in HIV and rehabilitation. AIDS Behav. 14, 448–458 (2010).
- Tricco, A. C. et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 169, 467–473 (2018).
- Miller, E. & Colquhoun, H. The importance and value of reporting guidance for scoping reviews: A rehabilitation science example. Aust. J. Adv. Nurs. 37, (2020).
- O’Brien, K. K. et al. Advancing scoping study methodology: A web-based survey and consultation of perceptions on terminology, definition and methodological steps. BMC Health Serv. Res. 16, (2016).
- Tricco, A. C. et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med. Res. Methodol. 16, (2016).
- Colquhoun, H. L. et al. Scoping reviews: Time for clarity in definition, methods, and reporting. J. Clin. Epidemiol. 67, (2014).
11. Peters, M. D. J. et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst. Rev. 10, 263 (2021).