HOW WE DID IT
The Fund’s community listening campaign kicked off with its first “kitchen table conversation” in the Madison/East End area in March 2014 and concluded with a July gathering in Pigtown. Kitchen table conversations were the primary method of data collection for this study. Each conversation typically involved between eight and twelve people taking part in a facilitated discussion in an informal, community-based setting. To ensure a consistent experience for participants citywide, the Fund trained hosts and facilitators using a Conversation Toolkit that included a conversation guide and tips for facilitators to keep conversations focused and inclusive of all participants.
In addition to kitchen table conversations, the Fund collected supplementary responses through one-on-one interviews and an online survey. All told, 859 individuals from all 55 neighborhood clusters, or Community Statistical Areas, in Baltimore City engaged in discussion about a vision for City Schools and what the new CEO’s priorities should be.
We recruited hosts for each conversation from the neighborhoods where the conversations were to be held—through neighborhood associations, parent-teacher organizations, community schools coordinators, churches, and other community groups. Conversation hosts then recruited a majority of the participants for the kitchen table conversations through their personal contacts, organizational affiliations, church groups, or neighborhood listservs.
Individuals who participated in the one-on-one conversations were randomly selected and approached during neighborhood canvassing. The Fund also used an online survey to ask the same open-ended questions of residents who were not able to participate in conversations. Participants who responded to the survey received the link from a conversation host or read about it in the media.
The Fund’s community study team examined hundreds of hours of conversations and survey responses and conducted an ethnographic thematic analysis to identify key findings citywide. Ethnographic thematic analysis is a statistical approach to analyzing qualitative data. The ethnographic method allows for an inductive analysis of the themes that naturally arise in conversation rather than fitting responses into predetermined categories/themes.
Five people performed the analysis of these conversations with the team lead conducting checks at every step for data quality assurance. The inter-rater reliability statistic, a measure of agreement, validity, and consistency among community study team members, reached a high of .81. A reliability statistic of .81 is rated as “excellent agreement” according to UCLA qualitative researchers and indicates the highest quality of data was produced by the community study’s team through strict coherence to the proper protocols of qualitative data analysis.
In order to generate themes, the community study team transcribed and analyzed the conversations using mixed methods analytical software. Team members applied codes to statements in each transcript, one-on-one conversation, and survey response reviewed. These codes were typically short phrases that described the content of the statement. The team then grouped the codes into like categories and quantified them. In this way, we were able to identify findings from the conversations.