This week I attended a public engagement session for the independent review of Ontario’s assessment and reporting practices. Early on in the session it was evident that most of the attendees were frustrated with EQAO testing and how it has changed the culture of learning and schools in the province.

The facilitators tried, sometimes unsuccessfully, to move attendees past expressing frustration and toward suggesting solutions. If the review is going to be more than just an opportunity to vent, we must quickly start asking “how can we fix this?” and “what comes next?”.

In that vein, here are four practices, three of which are currently used in large-scale testing in other jurisdictions, that might address some of the concerns and start to change the culture that’s grown up around 30 years of EQAO. This isn’t an exhaustive list, there are other ideas, but they do address some of my main concerns with EQAO as it’s currently practiced in Ontario.

Respecting Professional Judgement

Carole Campbell, who is leading the assessment review, suggested on social media that Scotland’s new assessment system is worthy of review.

The Scottish National Standardised Assessments (SNSA) assesses students in four different years (grades 1, 4, 7 and the 3rd year of secondary). The test is administered online and takes, on average, 45 minutes. One interesting feature of the SNSA is that teachers decide when to administer the test.

Teacher’s can choose to administer the test at the start of the year so that have that information to inform their teaching practice, during the school year at a time that doesn’t disrupt the normal learning or at the end of the school year to determine of students have reached the expectations for their grade level.

Local Control

The US is the home of bubble and pencil test. US students are some of the most heavily tested in the world and standardized testing has been part of the American educational landscape for a long time. So it’s ironic that some of the most creative efforts to innovate and change how testing is done is coming from south of the 49th parallel.

Schools in Tacoma, Washington have locally developed goals for achievement in Academic Excellence, Partnership, Early Learning and Safety. These goals are extensive, have measurable targets and were developed in consultation with all the local stakeholders. They reflect the current state of the local schools and the values and expectations of the local communities they serve.

Imagine a system in Ontario where different schools have different goals and expectations. These goals are clearly articulated and publicly committed to. Doesn’t that make more sense? Goals that reflect the communities that each school serves and that give each school something to move towards. A public recognition that schools are unique and about more than just academics, but also about connections with parents, community and making students feel connected.

Performance Tasks

Schools in New Hampshire, under the PACE Program, are piloting a system where teachers create performance tasks to assess progress on statewide competencies. Teachers create tasks that test the competencies and reflect their local needs. The tasks are submitted to the state board for approval. Once approved, performance tasks are administered and evaluated by teachers from another school and the evaluations submitted.

Tenth grade math students at Sanborn High are challenged to design a community water tower and then create a proposal that could be presented to the town. Imagine, students demonstrating their learning in a meaningful way.

Respecting Students

One of the biggest problems with EQAO is the refusal to consider student voice. No matter what students want or think, their input is not valued in any way. They can’t even opt out of the test if they want.

I propose adding a choice to the tests. Several different types of questions or tasks where students can choose which one works best for them. Allow students to choose a fixed number of questions to ensure all the required expectations are covered. Show students that this is something being done with students, not to them

This strategy would be particularly important for students with learning exceptionalities. As Krista Sarginson has pointed out, designing a test for mainstream students, and then making it “accessible” misses the point.

Adopting these ideas won’t fix EQAO overnight, but they will signal a change that the culture of testing in Ontario is changing, that may be more transformative than any changes to the actual test.

“In today’s increasingly interconnected world, people are often required to collaborate in order to achieve their goals. But students still typically learn individually. Schools will need to become better at preparing students to live and work in a world in which most people will need to trust and collaborate with other people.”

So begins The Programme for International Student Assessment‘s (PISA) first ever report on collaborative problem solving. PISA acknowledges that collaboration is a crucial but under studied area and that there’s a “lack of internationally comparable data” about collaborative problem solving.

What did PISA’s study show?

  • Students in Singapore scored highest.
  • 28% of students are able to solve only straightforward collaborative problems
  • 8% of students are top performers.
  • Collaborative problem solving is positively related to Science, Reading and Math scores, but the relationship is weak.
  • Students in some countries (Aus, NZ, Japan, US, etc.) did better than predicted based on their Reading, Math and Science scores.
  • Girls perform better than boys

PISA Collaborative Problem Solving Rankings (2015)

Canada did relatively well on the test ranking 5th in the world, but essentially in a tie for 4th with South Korea, Estonia and Finland.

Some Questions

Methods

PISA defines collaborative problem solving as follows:

I immediately noticed the change from “individual” to “agent”. Agent? What’s an agent? A secret agent?

The use of “agent” is intentional. PISA measured collaborative problem solving by having students interact with a computer simulation in “cognitive labs”. The other “agents” students collaborated with aren’t actual people, they’re simulated humans.

Why? To “isolate the problem-solving ability of the student”. Because, as we all know, students collaborate best in a cognitive lab when they are isolated and working with simulated humans 🙄

This is bizarre. Collaboration is about working with people to solve problems. It’s messy, and hard to tell who is doing what and how. It’s a complex process that requires students to be flexible, make decisions as they go and adapt quickly.

Rather than acknowledge this and develop a method where they observe students actually working together PISA missed the boat. They’ve chosen to value what can be easily measured rather than figuring out how to measure what’s valuable.

This isn’t the first time PISA’s testing methods have been questioned. Some countries that participate in PISA tests (China) don’t randomly sample their student population. A random sample of Canadian students are compared against students from city states such as Hong Kong and Singapore. Imagine how different Canada’s scores and rankings might be if we could choose which city we wanted PISA to test.

There’s also some suggestions that in some countries only students from high performing private residential schools are to participate in PISA test. These are the kind of practices that make international school rankings almost meaningless.

Equity

Notable is the idea that girls are much batter at collaborative problem solving than boys. What are the implications of this for a diverse school population?

Collaborative problem solving needs to be a commonly used strategy in a diverse classroom. It’s something many girls are good at and something many boys need to be better at. Heterogenous groupings of students engaged in problem solving tasks will help students become better.

But collaborative problem solving shouldn’t be used exclusively. Students also need time to work independently. Teachers need to be aware of this and adjust learning strategies according to the needs of their students.

Media

I’ve been fascinated to see the lack of response to this report by Canadian media. This is especially noticeable when compared to the media response whenever math scores drop. The results of the Collaborative Problem Solving Report have been widely shared across the world. In contrast, at this writing, Canadian media outlets haven’t covered the performance of Canadian students on an international test.

Why? Why is Canada lauded around the world for our education system while within Canada media chooses not to cover some positive results?

Math Scores

After the last round of EQAO scores I suggested that part of the reason for stagnant math test scores may be a mismatch between instructional methods and testing methods.The results of the Collaborative Problem Solving Report support this contention. The report showed that the relationship between collaboration a math scores is weak. An education system that devotes considerable time to developing collaboration skills isn’t teaching those skills that lead to high math scores. Perhaps our stagnant math scores are a reflection of this.

Policy

The PISA report offers two main recommendations for policy which I agree with.

  1. We need to help develop collaboration skills in schools. A couple of ways we can do this is by ensuring our classrooms are diverse and providing opportunities to develop these skills. Physical Education is identified as an area that “provides natural opportunities to develop collaborative skills”.
  2. Developing positive relationships at school can help build collaborative skills. The role of school culture and relationships is key and the benefits for students, schools and education systems will be significant.

“Technology is anything that wasn’t around when you were born.” US computer scientist Alan Kay, 1989.

Earlier this week Brian Aspinall shared the Bloomberg editorial “Kick Mobile Phones Out of Class“.

Cellphones in schools is a subject I’ve discussed at length before and one that doesn’t seem to be going away any time soon. It’s a bellweather issue, an issue that indicates clearly where you sit on the educational spectrum. Do schools teach students “the rules” or help students learn effectively? Read More

It is well accepted that we have a crisis in children’s mental health in Ontario. Children’s Mental Health Ontario (CMHO) reports that the number of children waiting for mental health services in Ontario has almost doubled in the past three years and an 18 month wait list for services is common. Parents and professionals regularly tell me that children can wait up to 3 years for treatment.

The consequences are felt all across the province. There’s been a 54% increase in visits to the emergency department and a 60% increase in hospitalizations for children with mental health issues. This further stresses an already over-taxed health care system.

And it shows up in the classroom. CMHO chief executive officer Kimberly Moran said “When one child has a mental health issue in the classroom it has a profound effect on the other kids … When kids are waiting sometimes up to two years for treatment, imagine the impact in the classrooms, with their teachers, as they wait for treatment, it just doesn’t make sense”. Educators would agree. Read More

I’ve written about the “cooling effect” that an increased use of social media by administrators and school boards has had on educators. EduTwitter is increasingly a platform of self-promotion and is used less as a platform where educational ideas are discussed. Educators are using anonymous accounts to avoid reprisals from supervisors and discussions between educators are moving into private channels where they can speak freely. In the process, the opportunity to include divergent voices is lost.

During the October 7th ResearchEd conference in New York Tom Bennett met Canadian teachers who were forbidden from writing about education and from being anonymous on social media.

(Click on the tweet below to read the entire thread)

Australian teacher Greg Ashman followed up by reporting that teachers in Australian public schools teachers are also forbidden from commenting on education policy on social media.

This kicked off a round of discussions on twitter about how common this is and why. In addition several educators contacted me through private messages. They don’t feel safe even discussing the issue on a public platform. That’s alarming.

On September 27th, 2017 The Ontario College of Teachers released a “Professional Advisory – Use of Electronic Communication and Social Media“. The majority of the advisory focusses on the use of electronic communication to interact with students, but in the section titled “Act Professionally” teachers are advised to avoid criticizing anyone in the school community (How large is the “school community”? Does it include the Minister?).

While this is only advisory, and the consequences for criticizing an employer online aren’t specified, the implication to many teachers are clear.

Some teachers have shared stories with me about their experiences on social media and are willing to have them published anonymously. Anonymous because they’ve already been threatened with discipline and don’t wish to make things worse. But they also feel they’ve been treated unfairly, and that teachers have not only a right but a professional responsibility to express their opinion on education. They believe this right and responsibility is being suppressed.

Here are a couple of stories that were shared:

Name Withheld By Request

 

Name Withheld By Request

In most of the situations people shared with me school administrators acted to silence teachers because they were threatened by their opinions or worried that what a teacher wrote would cause problems for them with parents or the community.

Some teachers are convinced their social media activities are being monitored by their school board or their administrator.

Whether this is true or not, it indicates the level of fear that exists for teachers who use social media.

School communications departments aren’t tasked with supervising staff social media activity. No school boards have the capacity to monitor staff social media completely. Board employees who are on social media may encounter staff tweets or posts but they are not there specifically to monitor other staff.

A few days ago a colleague told me that his Vice Principal admitted that she regularly “checks” his tweets. It’s the word checks that’s problematic of course.

If administrators are on social media to interact, to engage and to learn then they are welcome to express their thoughts and opinions. But social media flattens hierarchies and strips away institutional power, which some find threatening.

My hope is that enlightened leaders will recognize that engaged passionate educators who are willing to speak out and challenge conventional wisdom are to be supported and encouraged, not suppressed. These are the educators who will change staff cultures and improve the quality of education for all students.

%d bloggers like this: