2023 International Joint Initiative for Research in Climate Change Adaptation and Mitigation: Merit Indicators for the Review of Applications
Using the Matrices
For each criterion, the reviewer is asked to consider relevant elements, which may include a few, several or all the elements outlined in the matrix, as well as others not listed. The matrices are intended to be used as a guide. Unless otherwise noted, a proposal does not have to be rated Exceptional against all elements to receive an Exceptional rating for the criterion overall. It is left to the discretion of the reviewer to balance assessments of individual elements and to provide an overall rating per criterion.
For the high-risk, high-reward and feasibility evaluation criteria, the matrices indicate four ratings: Exceptional, Very Good, Fair and Poor. However, members can use a seven-point scale in their assessments, selecting ratings that fall between the four described. The seven-point rating scale for the high risk, high reward and feasibility criteria is as follows:
- Exceptional (described)
- Excellent (between Exceptional and Very Good)
- Very Good (described)
- Good (between Very Good and Fair)
- Fair (described)
- Inferior (between Fair and Poor)
- Poor (described)
Criterion: Fit-to-Program
Projects must receive a Pass rating for this criterion to be considered for funding. Projects that meet the expectation for the call must pass both elements defined below. This Pass/Fail assessment considers whether the minimum requirements are met. Proposals will be assessed on how well both elements are addressed within the high risk criterion.
| Fit-to-Program | Pass | Fail |
|---|---|---|
| Focus | The proposed project directly addresses at least two of the key risks outlined in the competition overview. |
The proposed project does not address at least two of the key risks outlined in the competition overview. |
| Approach | The proposal clearly establishes that the group engaged in the research meets the definition of a vulnerable group. |
The proposal does not convincingly justify how the group engaged in the research meets the definition of a vulnerable group and/or no group has been engaged. |
Criterion: Interdisciplinarity and Trans-sectoral Research
Projects must receive a Pass rating for this criterion to be considered for funding. Projects that meet the expectation for interdisciplinarity with respect to the 2023 International Joint Initiative for Research in Climate Change Adaptation and Mitigation can be defined by elements including, but not limited to:
| Interdisciplinarity and Trans-sectoral Research | Pass | Fail |
|---|---|---|
| Intergration and perspective | Integrates information, data, techniques, tools, perspectives, concepts or theories from two or more disciplines or bodies of specialized knowledge. Explains how the disciplinary and sectoral perspectives, methodologies and techniques will be integrated, maintained and implemented to create interdisciplinary and trans-sectoral approaches throughout the project. The various disciplinary approaches and perspectives are fully integrated. The project is not an amalgamation of disciplinary-specific approaches. |
The interdisciplinary nature of the project is achieved through an amalgamation of projects/activities that are disciplinary. Does not adequately explain how the interdisciplinary and trans-sectoral approaches will be integrated, implemented and maintained throughout the project. |
| Team | The interdisciplinary and trans-sectoral approach is reflected in the team. |
The team does not reflect the expertise or sectors required to execute the interdisciplinary and/or trans-sectoral approach. |
| Project design | Designed from an interdisciplinary perspective. |
The project is an interdisciplinary component “added on” to a more conventional project or program of research. |
Criterion: Equity, Diversity and Inclusion in Research Practice
To meet the program’s expectations for equity, diversity and inclusion in research practice (EDI-RP), projects must pass each of the following elements.
| Equity, Diversity and Inclusion in Research Practice | Pass | Fail |
|---|---|---|
| Analysis of context | Shows understanding of EDI considerations/systemic barriers in the context of the research team. Concrete and specific examples are cited in analysis. Demonstrates a strong commitment to EDI overall. |
Analysis of context is generic and/or does not point to one or more systemic barriers. Evidence of commitment to EDI overall is lacking. |
| Concrete practice for each area | Lists at least one concrete practice that targets the specific context listed for each area. |
A concrete practice is not listed for one or more of the areas, or the concrete practices listed are not related to the context that was described. |
| Implementation | Describes how the concrete practice has been/will be realistically implemented. Careful thought has been given to inclusion considerations in the recruitment, training and mentoring plan. Efforts to establish a diverse team, inclusive working environment and equitable opportunities for all team members are apparent. Strong evidence of accounting for members’ needs and engaging support systems (as necessary) to ensure equitable contributions of each person. |
Provides no or an unclear description of how the concrete practice will be implemented. The implementation plan is unrealistic or does not offer a well-thought-out plan to address inclusion considerations. |
| Impact | Explains how the concrete practice will impact EDI and describes an appropriate methodology for measuring success, including specific stepwise and overall evaluation criteria. |
Gives no indication of how the impact will be measured. Does not explain the anticipated impact of the concrete practice on EDI or how it will be measured. |
Criterion: High Risk
Projects that meet the expectation for high risk with respect to the 2023 International Joint Initiative for Research in Climate Change Adaptation and Mitigation can be defined by elements including, but not limited to:
| High Risk | Exceptional | Very good | Fair | Poor |
|---|---|---|---|---|
| Focus | A fully integrated approach that directly addresses two or more of the representative key risks while also considering, in a holistic manner, the effect the project outcomes may have on other risks. |
A fully integrated approach that directly addresses at least two of the representative key risks. |
Approach addresses two of the representative key risks but the level of integration in addressing the risks is somewhat lacking. |
Approach does not adequately address at least two of the representative key risks and/or addresses them independently without adequately considering or addressing the connections between them. |
The proposal clearly establishes that the group engaged in the research is among the most vulnerable groups according to the definition of a vulnerable group: being both physically and socio-economically vulnerable, as well as currently impacted by the effects of climate change. |
The proposal establishes that the group engaged in the research is a vulnerable group according to the definition: being both physically and socio-economically vulnerable, as well as currently impacted by the effects of climate change. |
The proposal establishes that the group engaged in the research has some level of physical and social vulnerability but is not among those currently most impacted by the effects of climate change. |
The proposal does not convincingly justify how the group engaged in the research meets the definition of a vulnerable group. |
|
| Novelty of approach | An exceptionally innovative approach, building on the latest methods, concepts, information and techniques. |
A significantly innovative approach that incorporates advanced techniques and methodologies from several disciplines. |
Represents a well-thought-out approach that is an innovative variation on a conventional approach. |
Represents a conventional approach to the problem. |
| Disciplinary and sectoral expertise | International team that fully integrates, in a manner that can be a model for other projects, expertise and perspectives from a variety of regions, disciplinary backgrounds and sectors, as appropriate, to meet the objectives of the project. |
International team that integrates expertise and perspectives from a variety of regions, disciplinary backgrounds and sectors, as appropriate, to meet the objectives of the project. |
Team includes expertise and perspectives from more than one region, disciplinary background and/or sector, but the level of integration between the perspectives is not evident. |
Team is missing expertise and/or perspectives that would have made the project approach more novel, and the outcomes more impactful. |
| Contextual expertise | Members of the vulnerable group are fully integrated into the team in a novel or transformative way, reflecting the co-development and co-creation approach. |
Members of the vulnerable group are active participants in the project and were involved in its design. |
Members of a vulnerable group are engaged with the project team and were consulted in its design. |
Members of a vulnerable group have not been adequately engaged with the project team or involved in the project design. |
| Scientific rigour | Research objectives, design, methodology and proposed analysis approaches are clearly articulated, comprehensive and follow best practices that will ensure high quality, reproducible research results. | Research objectives, design, methodology and proposed analysis approaches are clear and comprehensive, and mostly follow best practices that will ensure high quality, reproducible research results. | Research objectives, design, methodology and proposed analysis approaches are lacking clarity. It is difficult to assess if few best practices for high quality research will be followed. | There is a lack of clarity regarding the quality and rigour of the project. |
| Cogent (or persuasive, plausible) |
Highly compelling case for why the approach will successfully address two or more of the key risks outlined in the competition overview. | Compelling case for why the approach is plausible and addresses two or more of the key risks outlined in the competition overview. | The application is somewhat persuasive in terms of the potential of the approach to address two or more of the key risks outlined in the competition overview. | The application is not convincing in terms of the potential of the approach to address two or more of the key risks outlined in the competition overview. |
| Note: The viability of the project plan is assessed under the Feasibility criterion. The focus of this element is on the persuasiveness of the argument. | ||||
Criterion: High Reward
Projects that are high reward are those with the potential to contribute to mitigation and adaption responses to climate change for those most affected by climate change and can be defined by elements including, but not limited to:
| High Reward | Exceptional | Very good | Fair | Poor |
|---|---|---|---|---|
| Impact | Very significant scientific impact. |
Significant scientific impact. |
Notable scientific impact. |
Limited scientific impact. |
Very significant economic, artistic, cultural, social, technological or health impact. |
Significant economic, artistic, cultural, social, technological or health impact. |
Notable economic, artistic, cultural, social, technological or health impact. |
Limited economic, artistic, cultural, social, technological or health impact. |
|
The substantial change that is likely to result is clearly defined and specific. There is a solid plan to measure the impact, with a focus on implementing, testing and refining solutions to maximize potential impact. The significance of the stepwise and overall change is clearly described. |
The substantial change that is likely to result is clearly described. There is a sound approach to measure the impact. An implementation, testing and refinement of solutions to maximize impact is planned, although limited in scope. The significance of the overall change is described, but there is lack of detail regarding the stepwise approach. |
The substantial change that is likely to result is described in broad terms. There is a vague approach to measure the impact. An implementation, testing and refinement process to maximize impact is mentioned but not well described. The significance of the change is broadly described. |
There is lack of clarity or specificity about the substantial change that is likely to result. The proposal lacks a plan or approach to measure the impact. There is no implementation, testing and refinement processes planned to maximize impact. The significance of the change is not sufficiently articulated. |
|
| Reach | Major impact on a single or small number of groups, communities or subpopulations, with lessons for other contexts and strong potential for impact on large or multiple communities. The project is likely to produce effective strategies related to policy, communication and community engagement, which will encourage implementation in other groups, communities or subpopulations. |
Substantial impact on a single or small number or groups, communities or subpopulations, with lessons for others and potential to impact large or multiple communities. There are plans to develop strategies related to policy, communication and community engagement, which should help implementation in other groups, communities or subpopulations. |
Limited impact on a single or small number of groups, communities or subpopulations, with either limited or no lessons for others, or limited impact on multiple communities. There is mention of plans to develop strategies related to policy, communication and community engagement, but the plans lack details. |
No meaningful impact on any groups or communities, with limited or no lessons for others, and no impact on multiple communities. There is no mention of plans or means to investigate and develop strategies related to policy, communication and community engagement. |
| Likelihood | There is a strong likelihood that the significant impact will be realized. Potential for knowledge uptake and use is high, as evidenced by thoughtful consideration and involvement of potential users in the co-creation of the research. |
There is a reasonable likelihood that the significant impact will be realized. Knowledge uptake is likely. Potential users are involved in the project. |
There is a chance that the significant impact will be realized. The potential for knowledge uptake is limited. |
It is unlikely that the significant impact will be realized. The potential for knowledge uptake is low because there is insufficient engagement of potential users. |
| Short-term benefits | Short-term benefits are significant, clearly defined and specific, and a detailed measurement plan is in place. |
Short-term benefits are notable, clearly described, and a detailed measurement plan is in place. |
Short-term benefits are described and will be measured, but the measurement plan lacks details. |
There is a lack of clarity or specificity about the short-term benefits that will result. |
Criterion: Feasibility
A project’s feasibility can be defined by elements including, but not limited to:
| Feasibility | Exceptional | Very good | Fair | Poor |
|---|---|---|---|---|
| Value for money | The proposed project is excellent value for money, proposing efficient and effective use of resources to achieve the intended impact. |
The proposed project is good value for money, although lacking in some detail to assess the efficient use of resources. |
The intended impact(s) of the project are acceptable in relation to the level of resources requested. |
The intended impact of the project is deemed insufficient for the level of resources requested. |
| Building on current knowledge or prior art | The application shows that the research team has strong awareness of current and relevant research and prior art or knowledge. |
The application shows that the research team is aware of most current and relevant research, and prior art or knowledge. |
The application shows that the research team lacks awareness of relevant research or prior art or knowledge in one aspect/discipline related to the project. |
The proposed project does not seem to consider current and relevant research and prior art or knowledge (for example, by proposing approaches that have been tested and failed). |
| Work plan | The proposed work plan, including the methodological approach, is well described, reasonable and likely to be achievable within the proposed time frame. |
The proposed work plan, including the methodological approach, is described, reasonable and likely to be mostly achievable within the proposed time frame. |
The proposed work plan is reasonable. The methodological approach is lacking detail. The project objectives might be met within the proposed time frame. |
The proposed work plan is not reasonable/feasible. The methodological approach is missing or flawed. It is unlikely that the project objectives will be met within the proposed time frame. |
| EDI considerations in research design (EDI-RD)* | EDI-RD considerations have been integrated into the methodological approach. The impact on the methodological approach and/or design has been clearly described. |
EDI-RD considerations have been integrated into the methodological approach. The impact on the methodological approach and/or design has been described. |
EDI-RD considerations have been integrated. The impact on the methodological approach or design has not been described. |
EDI-RD considerations have not been integrated into methodological approach or design. |
| Approach to co-productionFootnote * | Co-creation, co-leadership and co-ownership with the vulnerable group(s) is clearly integrated in the project’s design. The methodological approach and/or theoretical framework reflects all the following key principles:
Attention to equitable processes and procedures for fair and respectful inclusion of the vulnerable group(s) and their perspectives is evident. |
Active engagement and reciprocity with the vulnerable group(s) are present and clearly described. Key considerations of SSHRC’s Merit Review of Indigenous Research have been incorporated into the project.The methodological approach and/or theoretical framework of the project reflect all four key principles. |
There is modest engagement and reciprocity with the vulnerable group(s). The methodological approach and/or theoretical framework do not adequately reflect all four key principles. |
Engagement and reciprocity with the vulnerable group(s) appear to be lacking or have not been described. The methodological approach and/or theoretical framework do not reflect incorporation of the four key principles. |
Note: These principles for co-production are based on SSHRC’s Guidelines for the Merit Review of Indigenous Research. While the principles can be applied to co-production approaches with any vulnerable group, they are essential when working with Indigenous Peoples. The guidelines can be consulted for further description of the principles and how they can be incorporated into the project design. |
||||
| Positioning for uptake | The research is designed with use in mind, with a very well-considered and detailed plan for engagement with stakeholders and end users (including ensuring findings are accessible and user-friendly). |
The research is designed with use in mind, with a plan for engagement with stakeholders and end users (including ensuring findings are accessible and user-friendly). |
There is engagement of potential users in the project but no plan to ensure uptake. |
There is no engagement with potential users and/or no plan to ensure uptake. |
| Environmental footprint | A thorough assessment of the potential environmental impact of the project has been completed, and a sound approach to minimize the environmental footprint is in place. |
An assessment of the potential environmental impact of the project has been completed, and some consideration as to how to minimize the environmental footprint is evident. |
A limited assessment of the potential environmental impact of the project has been completed, and/or little consideration as to how to minimize the environmental footprint is included. |
No assessment of the potential environmental impact of the project has been completed and/or no consideration has been given as to how to minimize the environmental footprint. |
| Research team | The application clearly demonstrates that the research team has the required expertise in all relevant disciplines to meet the objectives. |
The application demonstrates that the research team likely has the required expertise in all relevant disciplines to help meet the objectives. |
The application demonstrates that the research team has most of the required expertise, although some aspects may be missing or insufficiently described. |
The application does not clearly demonstrate that the research team has all the required expertise to complete the work. |
| Suitability of the research environment | The research environment is enriching for and provides numerous opportunities and support for the next generation of researchers. |
The research environment is enriching for and provides some opportunities for the next generation of researchers. |
There are limited opportunities for trainees to participate and/or the research environment is suitable but not enriching for the next generation of researchers. |
There are few opportunities and/or inadequate support for the next generation of researchers to participate and/or the research environment is not suitable for their involvement. |
| Management | There are sound plans in place related to:
|
There are sound plans in place related to most of the following:
|
There are sound plans in place related to some of the following:
|
There are a lack of sound plans in place related to:
|
| Resources | The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. All aspects have been described. |
The research team has acquired or has concrete plans to acquire the necessary resources to complete the work. Some aspects have not been well described. |
The application demonstrates that the research team has acquired or has concrete plans to acquire most of the resources to complete the work. Some aspects may be missing or insufficiently described. |
The application does not clearly demonstrate that the research team has acquired or has concrete plans to acquire the necessary resources to complete the work. |
Page details
- Date modified: