70 Community Engagement and Collaboration | Introduction to some early recommendations for tracking and evaluating community engagement within KMb 2

Background

Following the above resource, conceptually, evaluation is similar to validity testing in statistics. When you are looking at validity you are checking the degree to which a measure measures what it is supposed to measure, and when you are evaluating your community engagement you are checking the degree to which your engagement efforts had the impact that you intended and that those with whom you were engaging felt authentically involved in the process.

The Ontario Centre of Excellence for Child and Youth Mental Health has developed six key considerations for evaluating KMb, but these considerations also apply to community engagement work. Take a moment to review these six considerations below:

When reviewing the six considerations above, what comes to mind around how you could evaluate your community engagement work?

Theories and guiding principles for evaluating community engagement

There are various theories around the ways to evaluate community engagement and numerous indicators to choose from when evaluating, but one of the most important pieces to remember is to collaborate and communicate with your community partners when working towards an evaluation. For example, South and Phillips (2014) examined community engagement within the public health system and found that there may be numerous challenges in the “measurement” of community engagement and that evaluations can sometimes oversimplify the complexities of community engagement work.

When developing and coordinating an evaluation of community engagement it is critically important that researchers and their community partners work together in a systems approach – and that evaluations should be done with, not on, community partners/stakeholders. South and Phillips (2014) put forth the following eight ‘guiding principles’ that may be important to consider within evaluation of community engagement (noting that this work has public health lens, but that the implications are relevant for broader KMb work as well):

1. Communities should be considered an integral component of health systems.
2. Communities should be involved in identifying appropriate outcomes and defining success.
3. Evaluation should not seek to control complexity because community engagement approaches are complex, dynamic interventions.
4. Evaluation should be sufficiently flexible to measure unanticipated effects.
5. Evaluation needs to build a thick description and explanation of the nature of participation.
6. Where quantitative methods are used, social indicators that track changes in health determinants, including social structures, need to be given equal weight to individual behaviour change.
7. The purpose of the evaluation should be clearly defined in relation to the information needs of different stakeholders: policy makers, professionals, academics and communities.
8.  Funding streams need to shift to encompass funding of whole system evaluations.

Clearly evaluation of community engagement within community engagement and KMb is complex, so how can one set out to run an authentic and accurate evaluation of their engagement collaborations? What resources are available to support this type of evaluation work? Several of the KMb toolkits and planning templates outlined throughout this course highlight the importance of evaluation within KMb and present some indicators that may be relevant for your work. Below we have curated a non-exhaustive list of resources that may support your evaluation plans around your community engagement work.