7.5. Evaluation Methods

In previous sections, we discussed a couple of styles of feedback gathering: surveys and focus groups. There are other methods used to gather feedback, including interviews (one-on-one conversations with stakeholders), observation (watching the event and taking notes of what you see), and social media monitoring (what is being posted on social media channels about the event). The point of this chapter is to explore the pros and cons of each of the methods so that you can start to determine which combination of styles might be the best fit for your events.

Surveys

Surveys can be completed in a variety of ways (paper/pencil, over the phone, digitally) and offer a structured method of gathering feedback. Everyone gets the same questions that are chosen specifically by the evaluator. Each option can be used to reach a large audience – you can mail the paper versions, phones work around the world, and digital versions can be shared in many ways. Their anonymous nature allows folks providing feedback to potentially feel a little more comfortable answering honestly. However, they have some drawbacks as well, including potentially low response rates if the survey is too long, too general, or given to them at an inopportune time. Creating the perfect survey question often takes a lot of time, so evaluators may not feel it is worth their time. What experiences have you had with surveys? What would you add to this list of pros and cons?

Interviews

Because interviews are usually conducted in a personal, one-on-one setting, they allow for a more in-depth exploration of a participant’s experience. The interviewer can direct follow-up questions based on the participant’s answers. The biggest negative aspect of interviews is likely the time it takes to conduct them. They are time-consuming to conduct and to summarize. Additionally, participants may be less comfortable offering honest feedback because the interview is not likely to be anonymous. Finally – if the interviewer is not properly trained, they may make the mistake of guiding a participant’s response in one direction or another, which would skew the feedback results.

Focus Groups

Unlike interviews, focus groups are conducted in groups – the size of the group is determined by the person seeking the feedback. The groups can provide a diversity of answers that is not always possible in other settings. Folks offering feedback in groups often feel emboldened to share more readily in an effort to get their voice heard, especially if there is a feeling of comfort around collaboration to build on each other’s feedback. For instance, when surveyed about a recent experience watching a television pilot, one participant may not have noticed the music in the background until another participant mentioned it as pleasant. That comment spurs others to offer feedback about the music that the organizer may not have even thought to ask about. On the other hand, however, sometimes there can be a dominant voice in a focus group who speaks up so much that others do not feel comfortable offering their thoughts – either because that person is voicing similar feelings or the louder person is overpowering and the quieter person doesn’t feel as if their opinion is valued in the same way. It will be important for the facilitator to work to make sure all participants in a focus group have an equal chance to speak if desired.

If you’d like to learn more about how to organize and run focus groups, this article from Masterclass does a great job of clearly outlining the steps.

Observation

If there is time and the space allows, observation can be a very powerful feedback tool. Observers can capture an overall vibe at an event by watching how people interact with each other, how they flow through space, what foods are going back into the kitchen cold versus which trays are going back empty. They can also make note of non-verbal responses that attendees are making to the speakers to supplement the qualitative feedback provided using other methods. Unfortunately, sometimes, personal bias can overshadow a true representation of what is happening, and confirmation bias might occur. Confirmation bias is the tendency to interpret new information as confirmation of one’s existing beliefs or theories. In this case, an example of confirmation bias might be found if someone is observing a speaker who is talking about a topic that the observer finds confusing. When they observe folks in the audience to gauge their response, the observer might make a note of all of the attendees who have confused looks on their faces or look bored. That would confirm their thoughts that the speaker is boring and uninteresting.

Social Media Monitoring

Social media monitoring is an opportunity for event organizers to get a real-time response from attendees as they capture the engagement of attendees on a variety of social media platforms. If organizers create and advertise a common hashtag for folks to use when posting about the event on their personal platforms, they will get a broad sense of both positive and negative feedback from attendees. The negative side of using social media monitoring as a feedback source is that the posts may lack depth. Additionally, negative feedback on social media may cause detrimental consequences for the organizer, especially if the poster has a large and loyal following. Monitoring all of the social media platforms used by attendees can be difficult, so a lot of thought should go into whether or not this is an appropriate avenue for feedback for your event.

By understanding the pros and cons of each of the methods described above, event organizers can work with their teams to select the most appropriate approach based on the objective of the event, the resources available to them, and the preferences of all stakeholders. Sometimes, a combination of evaluation methods might be the most comprehensive path to gaining actionable insights for improving future events.