This week EQAO released its review of provincial testing scores and it again raised questions about whether or the relatively poor performance of students on math is due to the curriculum or the test.

Screen Shot 2017-08-31 at 8.40.15 AM

This was different by the way than the interpretation that EQAO put on the results. They said the that Grade 6 results were “stable’ with a small decline (1%) at Grade 3. (Incidentally reading scores either went up or stayed the same and writing went down by 1% but apparently we don’t care about literacy any more). Read More

Earlier this week, in response the parent and staff concerns, Earl Grey Senior Public School in The Toronto District School Board banned cell phones from class. This closely follows a similar move by Herring Cove Junior High School in Nova Scotia. At least, that was the media story.

In fact, the so-called ban was really just providing a set of restrictions around when students could and could not use their personal devices. Students could still bring their phones into class if their teachers wanted students to use them. Otherwise, they should be stowed in their lockers.

Students will be allowed to use their phones at lunch, but not for social media, texting, taking or viewing photos and videos. So I guess as a paper weight? And I wouldn’t want to be the staff who are responsible for supervising students who are on their phones at lunch and making sure they aren’t texting or posting. How could you ever enforce that?

The story did provide an opportunity to discuss the issue however. In my opinion the cell phone ban, and how people react to it, it a “bellwether issue”. If you see schools as places where students go to learn how to behave and follow the rules, you support the ban and want those distractions out of schools. If you see schools are places that engage students in learning about things they care about, you oppose a ban. I fall into the later category.

I was asked to speak about the issue with Wei Chen on CBC’s Ontario Morning (Feb 24, 2017) and you can listen to a “dodgy” recording of the interview below.

A few days later (Feb 28, 2017) I was also invited to speak about Cell Phone Bans with Matt Galloway on CBC’s Metro Morning.

And then on March 14th I was the guest on Ontario Today, Ontario’s No. 1 call-in radio show, to discuss Cell Phones in Class: Are they a hassle or a necessary tool?

This is an issue which resonates with a lot of people for a lot of different reasons. It says a lot about what we think schools are for, what learning in our classrooms should look like and what the influence of technology is or should be.

Your thoughts and feedback on the issue and the interviews are very much appreciated.

In 2013 The Ontario Public School Boards Association (OPSBA) made it a priority to develop a position paper on EQAO testing in response to Provincial Policy Memorandum 155 and the position paper of the Ontario Teachers’ Federation (OTF). A survey was distributed to trustees and senior staff of all the member boards and based on that some recommendations about EQAO were developed. The OPSBA discussion paper: “EQAO and Large Scale Testing In Ontario” was released to this public on December 12, 2016.

The paper has three main sections. The survey results, the recommendations and “Key Themes” from the survey,

Here are some highlights:

The Survey


Perhaps the most stunning finding of the survey is right up front. The survey found that 76.7% of OPSBA Trustees and senior staff want to either eliminate EQAO or agree that changes are required. Putting that in EQAO parlance only 23.3% of those OPSBA surveyed found EQAO to be at or above the provincial standard.


There are seven main recommendations:

  1. Equity & Accessibility: Work to remove cultural biases and make the test more accessible to students with IEPs.
  2. Technology & High Order Thinking Skills: Accelerate the use of technology in testing and have fewer multiple choice questions. Make the test more like the kind of learning that happens in the classroom.
  3. OSSLT: Allow students to go directly to the OSSLT remedial course without failing the test first. Make the OSSLT writing activities more like the writing students regularly do (collaboration, peer editing, etc.)
  4. Diagnostic Assessment: EQAO isn’t the same as a diagnostic test and shouldn’t be used that way.
  5. Student Well-Being: There was some suggestion in the Ministry’s Well-Being Strategy document that EQAO would develop a measure of student well-being. OPSBA doesn’t support this.screen-shot-2016-12-13-at-8-04-49-pm
  6. Overhaul how EQAO Shares Information With The Public:
    1. Clarify that EQAO data is limited in scope and has a narrow, specific purpose.
    2. EQAO should fit into school routines, and shouldn’t be a source of stress.
    3. Find a way of sharing data that prevents ranking schools.
  7. Use of Randomized Testing: The Ministry should examine whether randomized testing could effectively replace EQAO’s current Large Scale Testing model.


Overall this is a very exciting document. It’s wonderful to have OPSBA confirm and agree what many vocal critics of EQAO testing have been saying. It seems that the tide on standardized testing is truly starting to turn and it’s great to see OPSBA also calling for significant reforms along with Michael Fullan and Andy Hargreaves.

  • The failure of the OSSLT trial earlier this year has left many questions about EQAO’s ability to successfully manage this transition. EQAO needs to pull back from integrating technology into their testing and develop a more effective approach. Technological failures are further undermining confidence in their competence.
  • The hand wringing about Ontario students and their math scores continues. There’s every reason to wonder if part of the drop in student’s math scores is related to EQAO tests not effectively measuring the kind of math learning students are doing. Standardized tests are good at measuring procedural math but not very good at assessing mathematical thinking. Having EQAO more closely reflect classroom learning may well help with this discrepancy.
  •  The notion that EQAO would develop a Student Well-Being measure is disturbing. Students need less testing, not more. One of the most effective ways of enhancing student well-being would be to eliminate EQAO completely, so eliminating the associated stress for students.
  • It’s great to see OPSBA agree that EQAO must take steps to prevent the harmful misuse assessment data by other groups. EQAO’s stance has been to condemn the practice but to take no steps to prevent it. OPSBA recommends that they start to do that. How about collecting school specific data but not releasing it publicly? EQAO can still fulfill its mandate by publicly releasing results for the school board without stigmatizing low scoring schools.
  • I first heard Dr Joel Westheimer suggest that EQAO could move to randomized testing and get equally reliable data on student achievement. Doing so would also de-emphasize the number of students affected and the stress caused by EQAO. Schools wouldn’t know who was being tested, so no need for a lot of teaching to the test. Great to see that OPSBA agrees.


As previously discussed EQAO has announced that the technical failure of the online OSSLT was due to “an intentional, malicious and sustained Distributed Denial of Service (DDoS) attack—a type of cyberattack.” However some evidence has surfaced that contradicts those claims.

ICT Educator Elliot Royle tweeted  a link to the Digital Attack Map for October 20, 2016.

The Digital Attack Map “presents data gathered and published by Arbor Networks ATLAS global threat intelligence system” and shows all the DDoS attacks that happen around the world on any given day. It shows the source, target and duration of any DDoS attacks on that day.

A typical day like October 19th looks like this:

As you can see there were three DDoS attacks that day, targeting Brazil, The US and Great Britain. As is typical the sources of the attacks are routed through countries all around the world.

Here is the Digital Attack Map for October 20th, the day EQAO said the OSSLT was taken down by a DDOS


As you can see this map looks very different. There is some distributed traffic and an attack centred on The USA, but no attacks converging on Canada. Not one.

As another point of comparison, here’s the Digital Attack Map for October 21st, the day of the massive Dyn DDoS attack, the largest ever (so far):

Again, this looks quite different than the activity of October 20th when the OSSLT was cancelled.

So what can we make of this? There are only a few options.

The Digital Attack Map may be wrong. Arbour admits that while the “data represented in the Digital Attack Map is sourced from one of the most complete data sets available, it is an incomplete picture. The data may misidentify or exclude attack activity, and is intended to present high level trends in significant attacks as they are observed by Arbor Networks”.

There may have been an insignificant attack that doesn’t show on the map, but then EQAO’s system should have been able to withstand an insignificant attack.

Another option is that the failure of the online OSSLT wasn’t caused by a DDoS at all. This would be consistent with the reports by teachers that the MIST platform failed when it was tested last March and it crashed when it was used by Tennessee schools in February.

Perhaps EQAO can explain why Arbor Networks didn’t detect a DDoS attack on the day they said one took down the OSSLT?

After some trials in March of 2016, EQAO scheduled a large scale trial of The Ontario Secondary School Literacy Test (OSSLT) on October 20th and was set to accommodate up to 250,00 students in two sittings. It was a no-lose proposition for any students that chose to write it, because if they passed the online version they could avoid writing the test in its regular pencil and paper version in March, 2017. If they failed, they could still write it again later.

The day before EQAO was confident they were prepared for any potential technical issues:

But even as EQAO was wishing students good luck it became obvious they weren’t as prepared as they’d previously thought:

EQAO remained confident that this was a minor problem even though teachers assured them that it was anything but minor:

After an hour’s delay EQAO continued to insist that students wait in the assessment rooms:

It took almost 3 hours before EQAO announced (on social media) that they were cancelling the test:

At 6:23 pm on October 20th EQAO released an official statement apologizing for the cancellation, saying they didn’t know what had happened and would be investigating the cause. Over ten hours since they’d known there was a problem and EQAO still hadn’t figured out what happened.

There was no further comment from EQAO for four days, but On October 24th they announced that the test had been cancelled due to “…an intentional, malicious and sustained” type of cyberattack, a Distributed Denial of Service (DDoS) attack. Coincidentally, this was the same type of attack that took out large parts of the internet in Europe and North America on October 21st.

Since then the students and educators involved, and the province in general, have been trying to figure out exactly what happened. Education Minister Mitzie Hunter called it “extremely disappointing” and Education Critic Lisa Gretzky wondered about the costs of the failed test:

Around all this discussion is a sense of uneasiness. A feeling that we don’t have the full story about what really happened. There are so many unanswered questions.

For example:

Why did the EQAO attack last so long and take so long to identify?

The EQAO Attack lasted three hours and 10 hours later it still hadn’t been identified as a DDoS attack. By comparison the massive Dyn Attack was identified almost immediately, the public was notified right away, and it was resolved in a little over two hours. Why did it take EQAO so long?

Who investigated ?

Apparently “IT experts and a third-party forensic team spent the weekend trying to figure out what happened” but who exactly did the investigation remains a mystery. Why? When tax dollars are being used why isn’t there full and transparent disclosure?

What successful trials?

Before and after October 20th EQAO mentioned repeatedly that they had conducted “several successful tests” of the OSSLT platform. However, some of the teachers involved in those tests dispute that they were “successful”, and said that the same issues that shut down the OSSLT on October 20th were present in the March tests.

Was the testing platform stable?

The online platform that EQAO was using for the OSSLT is called Measurement Incorporated Secure Testing (MIST), sold by Measurement Incorporated, a company based in North Carolina that “provides achievement tests and scoring services for state governments, other testing companies, and various organizations”.

The same platform had problems in February, 2016 when used for school testing in Tennessee that seem awfully similar to what happened in Ontario on October 20th.

There are also many implications for EQAO moving forward:

  • While the test was eventually cancelled some students were able to complete the online test (why??). Will those completed tests still be marked? If they aren’t, isn’t that unfair to those students who persevered?
  • EQAO stores millions pieces of data on students from eight years of age and older. Given what happened on October 20th, how can Ontario parents remain confident that personal information about their children will remain private when testing becomes online?
  • Will we ever know what happened? Richard Jones, director of assessment for EQAO said “I’m not sure if this kind of thing can ever be figured out,” which won’t fill anyone with confidence.
  • Should EQAO have been better prepared for what happened. Cyber security lawyer Imran Ahmad thinks so:


Ontarians should know what happened, why it happened and what EQAO is doing to prevent it from ever happening again. Our students and families deserve that. A confused shrug from Richard Jones isn’t good enough.




%d bloggers like this: