EXPLORING NEW FRONTIERS IN ASSESSMENT AND LEARNING

Microphones and Gum Drops

September 13, 2019

Guest post from 2019 Summer Journeyer, Kate Moore.

As a Summer Journeyer, I experimented with different methods of capturing audio data during collaborative problem solving activities among middle school students. My goal was to figure out a way to capture what each student said while they worked with a small group of their peers to think through a complex problem. I am interested in the communication strategies students in this age group use to explain, propose, persuade, negotiate, etc. Ultimately, I hope to design a tool that can use audio data to measure collaborative processes in classrooms. I think this will be a powerful contribution to current efforts to change the way we measure and assess 21st Century skills in our classrooms.

How to actually do this is a big question. So far, my work at Teachers College, Columbia University has examined collaborative processes among adults in controlled laboratory environments. While our findings have been fruitful, we wonder if (or how) the collaborative processes we observe in lab settings differ from those of students in classroom settings. We expect results to be quite different. Middle school students are at an entirely different developmental stage than the adults we have worked with, so we don’t expect the training data that we have gathered from our lab studies to be particularly sensitive to student communication processes. This is why recording what students actually say in authentic settings is so important. We want whatever evaluation tool we develop to be trained on what the STUDENTS say, not the adults from our lab. We hope our methods will help design a tool that is a better “listener” to students. 

This summer, I was able to conduct several play tests of simple collaborative problem-solving games using a variety of microphones designed to minimize background noise (a major obstacle in this line of research). Initial findings suggest that different age groups of students use remarkably different communication processes (as expected). I also found that different types of collaborative tasks (e.g., making a list, designing a network, and constructing a model) generate slightly different communication patterns (e.g., more ideas proposed when making a list than when designing a network). Not only do these findings suggest that collaboration looks different depending on the task, but it also begs the questions, “What does collaboration even look like?”  and “How can we tell when students are collaborating… and collaborating well?”

This research will continue as I return home, back to Teachers College and New York City! Here, I will continue to develop a procedure / pipeline for gathering audio data and processing it in order to evaluate it. I will also keep working to train an algorithm based on this data so that, eventually, we might have a tool that is better tuned to what middle school students are saying. Big picture, I really appreciate connecting with the team at the Playful Journey Lab. I have learned a TON about the technicalities of recording audio and about the pipeline needed to automate the data collection process. It has been a fast and furious summer experience that has tuned and focused my future research. Best of luck out there Playful Journeyers! Many thanks!