Evaluation of the Social Sciences and Humanities Research Council Connection Grants

Summary

About the evaluation

This evaluation examined Connection Grants’ (CG) relevance and outcomes and was intended to provide recommendations for improvements. The study also included consideration of relevant changes outside of CG. CG was last evaluated in 2018 as part of an evaluation of the other Research Partnerships Program (RPP) funding opportunities, with which it shares common objectives. The current evaluation focused solely on CG and provided an opportunity to assess how CG is functioning, including its value to the research community and potential directions for the future.

The evaluation included multiple lines of evidence: analysis of agency administrative data related to research participation in CG and other SSHRC funding opportunities; qualitative analysis of funded and unfunded CG proposals and CG grantee achievement reports; key informant interviews with CG applicants, recipients and sponsoring organizations; a review of agency documents and academic literature on knowledge mobilization; and a focus group with Social Sciences and Humanities Research Council (SSHRC) staff. The evaluation was guided by an Evaluation Advisory Committee, in accordance with the federal Policy on Results.

About the funding

The CG funding opportunity sits within the RPP at the SSHRC. It provides grants of up to $50,000 for events and outreach activities geared toward short-term, targeted knowledge mobilization initiatives. Grants of higher amounts are possible if satisfactorily justified. The purpose of the funded activities is to offer opportunities for knowledge exchange and engagement with participants, specifically around research issues of interest to participants. CG therefore serves as a complement to other funding opportunities that fund research.

Between 2017 and 2022, CG funded an average of 305 initiatives per year for a total average investment of $7,887,232 per year.

Evaluation questions

1. Has the way that knowledge mobilization (KMb) is described at SSHRC changed since CG was launched? If so, in what way and what are the implications for CG?
2. Does CG continue to make a distinct contribution to SSHRC’s objectives for knowledge mobilization?
   1. To what extent and in what ways do initiatives funded by CG contribute to research funded by other SSHRC FOs, specifically by Research Partnership Grants (PG, PDG, PEG) and Insight Grants (IG, IDG)?
   2. To what extent does CG attract and fund proposals for knowledge exchange activities and events between groups of academics, as well as between academics and non-academics?
   3. To what extent does CG make other, distinct contributions of importance to SSHRC and the research community?
   4. Are the tools used for knowledge exchange by the research community changing? If so, in what way, and to what extent does CG support changing tools?
3. What opportunities are there, if any, to optimize CG’s role and contributions?

Key findings

Knowledge mobilization at SSHRC

• SSHRC’s public-facing descriptions of KMb have changed in subtle ways since CG was launched in 2011.
• Descriptions of KMb in the Definitions and Guidelines may not have kept pace with changes in SSHRC’s corporate priorities, such as its support for Indigenous research, or with developments in KMb scholarship.
• There appear to be inconsistencies in the way KMb is used in different public facing documents produced by SSHRC, and relative to specific initiatives. Focus group participants expressed that there may be inconsistencies in the ways different agency communications use the term, with some program language and merit review criteria equating it mainly with the dissemination of research results.

Contribution to other funding opportunities

• CG contributes to knowledge mobilization in a variety of ways, including by acting as a catalyst for new research and the initiation of partnerships and collaborations, by enriching concurrent research, and by supporting the dissemination of research results.
• A majority (but not all) of CG recipients have received research funding from SSHRC through IG or RP, and it is more common to hold a research grant prior to a CG than after. Between 2017 and 2021, 77% had participated in one or more research grant competitions prior to or after their CG.
• There is not always a direct correspondence between a CG and a research grant. Often, a CG-funded activity is contributing in more than one way to a research project, and/or to more than one project.

Academic and non-academic involvement

• CG attracts and funds more proposals for activities between groups of academics than between academics and members of other sector organizations. Participants from other sectors appear somewhat more often as collaborators than as co-applicants. Only 9% of sampled CG proposals included a co-applicant from another sector. One quarter (25%) listed a collaborator from another sector. In achievement reports, 69% reported other sector participation in a broad informal capacity.
• Other sector involvement in formal roles on a CG application was reported to be limited in part due to an application process that is designed for academics, a lack of resources to enable other sector participation, and a perceived lack of benefits for some types of other sector actors when they do participate.

Contributions to SSHRC and the research community

• CGs contribute to KMb-related outcomes in ways that would be difficult to replicate with other FOs. Elements of CG’s design that make it easily adaptable, such as its flexibility (e.g., frequent calls, diversity of eligible projects), relatively small dollar values and short durations, were described as valuable by interview participants. CG’s flexibility appears to enable it to fund a broad range of activities, and occasional adaptations, such as for special calls in areas of strategic importance for SSHRC. The Indigenous Research Capacity and Reconciliation special call illustrates the possibilities of adapting CG’s design elements when appropriate and necessary to aid in advancing priorities.
• Reported outcomes vary from immediate to long-term and can be far-reaching. CG-funded events and activities were reported to contribute to advancing knowledge in relevant academic fields within Canada and internationally, and in other sectors. Events and activities funded by CG do not only affect those who are directly involved but also have ripple effects that continue after the project is over, affecting other people and places, as reported by interviewees. Other reported outcomes include expanded networks, sharing research results, and developing KMb-related skills among trainees.

Support for tools for knowledge exchange

• Overall, there was minimal change in tools used for knowledge exchange in proposals for CG funding between 2012 and 2022. Traditional tools like conferences and workshops were most common, but a gradual reduction in their prominence is evident. Less established tools like online archives and summer training institutes were proposed and funded in small numbers but with increasing frequency.
• The COVID-19 pandemic accelerated a shift from in-person to online interactions in KMb. Interview participants described increased online outreach and collaboration driven by technological developments, with the COVID-19 pandemic as an event that disrupted the status quo and accelerated movement in this direction.
• Data from proposals and interviews indicate that written publications, such as journal articles and books, and face-to-face gatherings are still considered important.
• CG supports a variety of tools, both traditional and less common, and has opportunities to support newer and less common tools more explicitly. Interview participants made references to CG supporting a variety of types of events and activities with different formats, some of which would not easily find funding elsewhere.

Opportunities to optimize CG

• Inclusion of other sector partners on CG research teams can be facilitated by streamlining some aspects of application procedures and through exploring options to increase recognition of other sector partners for participation in CG grants.
• Grant amounts could be adjusted to reflect inflation. High levels of inflation, which rose during the pandemic and at the time of this evaluation had only recently begun to stabilize, have resulted in significant increases to event-related costs, including transportation, accommodations and catering.
• Changes to data collection methods for proposals and achievement reports could facilitate the use of program data in future evaluations.

Recommendations

Recommendation 1: Continue to provide funding for knowledge mobilization events and outreach activities through Connection Grants in multiple competitions per year to allow for flexibility. Consider adjusting grant amounts for inflation.

Recommendation 2: Continue to recognize and support the wide range of tools used in KMb with a wide range of purposes, including newer and less common options that have the potential to engage researchers, trainees, other sector partners and communities in meaningful ways.

Recommendation 3: Adopt the draft logic model developed during the design phase of the evaluation of the Connections Grants FO, with some adjustments.

Recommendation 4 (for SSHRC): Review and update SSHRC’s public-facing descriptions of KMb, especially the Definition of KMb and the Guidelines for Effective KMb, to ensure consistent framing and use of relevant terminology.

Recommendation 5: Seek opportunities to facilitate increased levels of formal participation in CG projects by other sector actors (i.e., as co-applicants, collaborators or partners) by clarifying program language and modifying application procedures for co-applicants. Consider ways to improve recognition of the time and work that go into some forms of participation in CG-funded activities.