“In today’s increasingly interconnected world, people are often required to collaborate in order to achieve their goals. But students still typically learn individually. Schools will need to become better at preparing students to live and work in a world in which most people will need to trust and collaborate with other people.”

So begins The Programme for International Student Assessment‘s (PISA) first ever report on collaborative problem solving. PISA acknowledges that collaboration is a crucial but under studied area and that there’s a “lack of internationally comparable data” about collaborative problem solving.

What did PISA’s study show?

  • Students in Singapore scored highest.
  • 28% of students are able to solve only straightforward collaborative problems
  • 8% of students are top performers.
  • Collaborative problem solving is positively related to Science, Reading and Math scores, but the relationship is weak.
  • Students in some countries (Aus, NZ, Japan, US, etc.) did better than predicted based on their Reading, Math and Science scores.
  • Girls perform better than boys

PISA Collaborative Problem Solving Rankings (2015)

Canada did relatively well on the test ranking 5th in the world, but essentially in a tie for 4th with South Korea, Estonia and Finland.

Some Questions

Methods

PISA defines collaborative problem solving as follows:

I immediately noticed the change from “individual” to “agent”. Agent? What’s an agent? A secret agent?

The use of “agent” is intentional. PISA measured collaborative problem solving by having students interact with a computer simulation in “cognitive labs”. The other “agents” students collaborated with aren’t actual people, they’re simulated humans.

Why? To “isolate the problem-solving ability of the student”. Because, as we all know, students collaborate best in a cognitive lab when they are isolated and working with simulated humans 🙄

This is bizarre. Collaboration is about working with people to solve problems. It’s messy, and hard to tell who is doing what and how. It’s a complex process that requires students to be flexible, make decisions as they go and adapt quickly.

Rather than acknowledge this and develop a method where they observe students actually working together PISA missed the boat. They’ve chosen to value what can be easily measured rather than figuring out how to measure what’s valuable.

This isn’t the first time PISA’s testing methods have been questioned. Some countries that participate in PISA tests (China) don’t randomly sample their student population. A random sample of Canadian students are compared against students from city states such as Hong Kong and Singapore. Imagine how different Canada’s scores and rankings might be if we could choose which city we wanted PISA to test.

There’s also some suggestions that in some countries only students from high performing private residential schools are to participate in PISA test. These are the kind of practices that make international school rankings almost meaningless.

Equity

Notable is the idea that girls are much batter at collaborative problem solving than boys. What are the implications of this for a diverse school population?

Collaborative problem solving needs to be a commonly used strategy in a diverse classroom. It’s something many girls are good at and something many boys need to be better at. Heterogenous groupings of students engaged in problem solving tasks will help students become better.

But collaborative problem solving shouldn’t be used exclusively. Students also need time to work independently. Teachers need to be aware of this and adjust learning strategies according to the needs of their students.

Media

I’ve been fascinated to see the lack of response to this report by Canadian media. This is especially noticeable when compared to the media response whenever math scores drop. The results of the Collaborative Problem Solving Report have been widely shared across the world. In contrast, at this writing, Canadian media outlets haven’t covered the performance of Canadian students on an international test.

Why? Why is Canada lauded around the world for our education system while within Canada media chooses not to cover some positive results?

Math Scores

After the last round of EQAO scores I suggested that part of the reason for stagnant math test scores may be a mismatch between instructional methods and testing methods.The results of the Collaborative Problem Solving Report support this contention. The report showed that the relationship between collaboration a math scores is weak. An education system that devotes considerable time to developing collaboration skills isn’t teaching those skills that lead to high math scores. Perhaps our stagnant math scores are a reflection of this.

Policy

The PISA report offers two main recommendations for policy which I agree with.

  1. We need to help develop collaboration skills in schools. A couple of ways we can do this is by ensuring our classrooms are diverse and providing opportunities to develop these skills. Physical Education is identified as an area that “provides natural opportunities to develop collaborative skills”.
  2. Developing positive relationships at school can help build collaborative skills. The role of school culture and relationships is key and the benefits for students, schools and education systems will be significant.