Category Archives: Research

Deeper Learning: Chapter 1 – Deeper Learning for Students Requires Deeper Learning for Educators

Screen Shot 2017-08-06 at 9.22.18 PM

I was so eager to read this chapter of our summer+ learning faculty reading from Deeper Learning: Beyond 21st Century Skills.  Something that I heard Dylan Wiliam say back at NCTM in April 2017 has stuck with me – “Teachers need a habit change more than they need more knowledge.” How do we move teachers from knowledge to action? I wish this chapter had been required reading for all of our Upper School teachers and not an optional one.  Excellent read and glad I chose this chapter.

Deeper Learning: The Foreword (part 1 of my notes)

Deeper Learning: The Introduction (part 2 of my notes)

Notes on Deeper Learning: Beyond 21st Century Skills – Chapter 1: Deeper Learning for Students Requires Deeper Learning for Educators by Richard DuFour & Rebecca DuFour

  • Where our education currently stands
    • Our education has gotten better than in the past, contrary to reports.
    • The percentage of high school graduates keeps increasing
    • More student are taking honors & AP classes.
    • However, a good high school education is only preparing our students to live in the 1960’s!
    • Yet, 37% of our student who enter college require remedial courses (p. 28)
  • Economic shifts
    • In 1970, 74% of the middle class was made up of HS graduates and dropouts.  By 2007, only 31% of the middle class was from these groups.
    • In the same time period, the middle class with postsecondary education went from 26% to 61%.
    • Between 2010-2020, only 7% of jobs will be available for HS graduate and dropouts; those are limited to either low paying or declining in numbers.
    • Postsecondary education & training is no longer just the preferred pathway to the middle class, it is increasingly the only pathway.
  • What this data means
    • All HS students need the deeper learning that used to only be required for our best & brightest. (p.23)
      • Response: A bit of a push back here.  Why have we allowed good job training in HS to basically go away? Where students could learn how to be a mechanic before they graduated and had a job lined up? I just don’t believe everyone needs to go to college or should go to college.  They do need either job training or college though.  I’d like to see that job training go back to being part of high school.
    • The most promising strategy is to develop the capacity of educators to function as members of collaborative teams within a high-performing professional learning community (PLC). (p.23)
      • Response: What makes a PLC “high-performing”? What kind of leadership is needed to make this happen?
  • What is a PLC?
    • “an ongoing process in which educators work collaboratively in recurring cycles of collective inquiry and action research to achieve better results for the students they serve” (DuFour, DuFour, Eaker, & Many, 2010, p.11).
      • Response: What if some the educators in the PLC are resistant to change? 
    • 3 big ideas drive PLC process
      • 1. All students learn at high levels. 4 critical questions
        • a. What is it we expect all students to know and be able to do as a result of each essential standard we teach?
        • b. How will we know when each student has learned?
          • Response: This is SO key! Formative assessment is to inform us, the educators, about next steps.  It doesn’t happen enough.
        • c. How will we respond when, at the end of a unit, some students have not learned?
          • Response: Great question.  I find this difficult in math because of the time pressure to keep moving on.  How can I circle back with those students who are struggling?  Or, will our newer method of lagging assessment help in this area so that all students have the time needed to master concepts?
        • d. How will we enrich and extend the learning for students who have demonstrated proficiency:
          • Response:  Another difficult one for me.  In the past, I had students complain that they had more work. Obviously I didn’t pick the right extensions or way to offer those extensions to the students.  I need to work on this.
      • 2. Collaborative teams of educators need to work interdependently to achieve shared goals for which members are mutually accountable.
        • Response: What if the goals aren’t shared? I’m assuming the goals are that of the particular school or school system. This can be tricky in the hiring process.  Some people are good in interviews but not so great as employees.  Other times you are desperate for a teacher in a high need discipline and don’t have a lot of options.  
      • 3. Educators use evidence of student learning to inform and improve their professional practice & to better meet the needs of individual students.
  • What is Deeper Learning (pg. 24-25)
    • “the process through which an individual becomes capable of taking what was learned in one situation and applying it to new situations (i.e. transfer)…by developing cognitive, interpersonal, and intrapersonal competencies.”
    • Recommend Webb’s modification of Bloom’s and using the 4 DOK levels
      • DOK 1 – Recall of a fact, term, concept, or procedure – basic comprehension
      • DOK 2 – Application of concepts or procedures involving some mental processing
      • DOK 3 – Applications requiring abstract thinking, reasoning, or more complex inferences
      • DOK 4 – Extended analysis or investigation that requires synthesis and analysis across multiple contexts and non-routine applications.
  • Projected increase in rigor on state standardized tests under the CCSS
    • All students will be assessed with items at DOK 3 & 4
    • 70% of items in mathematics will be assessed at DOK 3 or 4 levels
      • Response: This is HUGE! Lots of students pass math classes but struggle with these higher levels of DOK.  I go back and forth on this.  In one sense, I believe all students can learn math at higher levels & I want them to.  However, I also think students who pass with a basic understanding still passed and not sure that means they should be considered deficient on a standardized test.  As a private school teacher, I didn’t know this because our students don’t take these tests.
    • All students will be assessed at DOK 3 & 4 in language arts
    • 68% of items in reading will be assessed at DOK 3 & 4
  • Addressing Common Core through PLCs
    • They are organized by grade level or content.
      • Response: I love my school.  I wish we had content leads and more work in HS organized around content. I do interdisciplinary work as well & will continue to do so, but I have to know the math content and how to be a good math instructor.  I hope our school will organize PLCs around content in the HS (or around groups like STEM/STEAM & Humanities).
    • The deeper learning of the CCSS certainly lends itself to interdisciplinary efforts that cut across subject areas, but we recommend that these efforts come later in the process.  In the initial stages of teaching and assessing for deeper learning, job-alike teams represent the best structure for collective inquiry.
      • Response: YES! This is what I just said above.  I think in our quest to go interdisciplinary, we didn’t make sure that teachers were solid in their own content and pedagogy first.  Too many things at once overwhelm people.
    • Collective inquiry starts with why before the how.
  • Moral purpose is not enough
    • “the quality of an education system cannot exceed the quality of its teachers. The only way to improve outcomes is to improve teaching.”
    • If Ss are to learn at deeper levels, schools must create the conditions that allow for the ongoing, deeper learning of the educators who serve those students in each of the three critical areas  (1) curriculum, (2) pedagogy, and (3) authentic assessment.
      • Response: YES to both of these!!  Something I struggle with – it seems as if people think everyone in the room has an opinion of equal weight.  I don’t get this.  If I were a novice to something, say programming on a computer, and I were in a room of experienced programmers, I don’t think my voice/opinion should carry the same weight.  In teaching, I haven’t seen it work that way.  I’ve watched our teams make poor educational decisions because the “majority” were people who don’t have the same understanding of curriculum, pedagogy, and assessment as more experienced & more educated teachers.  This is nonsense to me. 
      • Edited to add:  I think the top thing our school needs to do to help in these 3 critical areas is to address the faculty turnover rate.  Our turnover rate is high and we don’t do exit interviews.  I understand exit interviews are like summative assessments, they are post-mortems, but I think they still need to be done.  More importantly, some type of “formative assessments” should be done by admin/leadership to determine where teachers are throughout the year.  This has to be done in a way where the faculty don’t fear for their jobs by being honest.  IMO, this is the #1 priority our school needs to work on.  The work we do yearly on hiring & then training new faculty is exhausting, time consuming, and drains resources.
    • Curriculum
      • Challenge – how to best ensure that each teacher is clear about and committed to teaching the essential standards.
      • It is not unusual to see a huge gap between the intended curriculum and the implemented curriculum
      • With this gap it is common to see 2 things:
        • students receiving vastly different curriculum based on their teacher
        • admin being very “top-down” and regulating every single thing.  this fails to generate the clarity, coherence, or commitment to the curriculum. No involvement = no commitment
      • The only way to guarantee a curriculum is if the teachers who will deliver the curriculum have worked collaboratively to:
        • study the intended curriculum
        • agree on priorities within the curriculum
          • Response: what if there isn’t agreement?
        • clarify how the curriculum translates into student knowledge & skills
        • establish general pacing guidelines for delivering curriculum
        • commit to one another that they will, in fact, teach the agreed-upon curriculum
          • Response: Again, what if teachers make this commitment to the team and then still do it different in their classroom because they didn’t agree on the priorities?
          • When done well, a team will regard the essential skills of a guaranteed curriculum…as a promise they are making to students and to one another about what each student will learn.
    • Pedagogy
      • “a bad curriculum well taught is invariable a better experience for students than a good curriculum badly taught: pedagogy trumps curriculum” ~ Dylan Wiliam
      • well designed, inquiry based problems & tasks
      • build on prior knowledge – scaffold
      • clearly defined standards of proficiency
      • students receive formative feedback as the move forward towards those clearly defined standards
      • students engage in frequent, small-group collaboration
      • teachers help students develop their meta cognitive skills
      • teachers of deeper learning expect students to access, evaluate, and explain competing information about issues; develop & respond to probing questions; work collaboratively; justify their answers
      • Inquiry based does not mean unstructured or minimal guidance (emphasis mine)
      • Inquiry based is still designed in a structured way to lead students to the learning goal – balance students’ need for direct instruction with opportunities to inquire
      • students must explicitly be taught norms that foster healthy, collaborative group work to function as a team
        • accountability for the team as well as individuals
      • no one can script instruction that is guaranteed to result in deeper learning
      • the ultimate test of effective instruction (using solid pedagogy) is actual evidence that students have learned
    • Authentic Assessment (while I love all 3 – curriculum, pedagogy/instruction, and assessment; assessment has always been my biggest interest)
      • widespread agreement that assessments must change if they are to promote & measure deeper learning
      • high quality assessment could serve as a catalyst for improved instruction because if “assessments measure deeper learning abilities, teachers are more likely to teach the relevant skills.”
      • collaborative teams must help each other develop excellent formative assessment skills
      • Dylan Wiliam’s Embedded Formative Assessment is an excellent resource for PLC teams
      • teachers can learn what they need to know about writing good assessments by reading an article or introductory text on the topic
        • Response: Pet peeve of mine, which relates to something I mentioned under pedagogy, why don’t schools of education focus more on teaching about how to do assessment?! In my 2 year MA in Education program – not one class on assessment.  In my 2 year Ed.S program – 1 class on assessment. It’s a topic I love, so I’ve read a lot of research, etc.  However, most teachers have had little to know learning on this topic.  They just assess the same way they were assessed.
      • They recommend looking at PARCC and Smarter Balanced tests to get an idea of how students will be formally assessed.  Since I’m at a private school our students don’t do this.  I do think it is important to take a look at SAT & ACT exams since most students still need to take one of them for college admission.
      • the effort to have high quality, authentic assessment will have no result on student learning unless educators use the results to respond to the needs of the individual students & to inform the educator’s professional practice
      • team created common assessments are a means to an end – they assess the agreed upon meaning of the standard/learning outcome
        • Response: what if only 1 teacher instructs a certain course? how to get the team together to write these common assessments? what if not all team members see this as necessary?
        • these common assessments are an instrument for identifying students who are unable to demonstrate proficiency – this helps students and teachers
  • When students don’t learn – in a PLC school, collaborative teams of educators analyze the evidence of student learning from frequent common formative assessments to ensure that students who are struggling receive additional time and support for learning that extends beyond the individual classroom teacher to a include a schoolwide plan of intervention.  This intervention must be:
    • timely
    • diagnostic
    • directive
    • fluid and flexible
    • systematic
      • Response: Amen & amen to this whole bit about supporting students. This reminds me of the recommendations in NCTM Principles to Action for additional support for students who need additional time/help.  It is the school’s responsibility to help students in these areas.  If we accept a student in our private school, we are saying they can be successful.  
  • Common Formative Assessment as a catalyst for professional learning
    • in a PLC, use the evidence of student learning as a feedback loop to analyze the impact of their professional practice
      • Response: In our school – how do we get this to happen? 
      • if the assessment reveals one colleague has exceptional results with a skill or concept, that person can share strategies with colleagues
      • if the assessment reveals one colleague has had difficulty to help their students reach proficiency with a skill, the team can brainstorm ways that the teacher can integrate new instructional strategies
    • **Using evidence of student learning to inform and improve professional practice is the sine qua non of a PLC.**
    • Hattie concludes that schools should have systems in place to ensure that:
      • educators are working as members of a team & not in isolation
      • there is a shared understanding of the knowledge, skills, and dispositions all students must acquire
      • evidence of student learning is collected in a regular & dependable way
      • students are provided with multiple opportunities to demonstrate their learning
      • educators use the evidence of student learning to examine their teaching
  • Key to helping students is to invest in ongoing professional development
    • educators must assess the quality of their instruction on the basis of actual evidence from team created common assessments
    • when a team discovers none of its members are being effective in helping students with a concept, then they must seek assistance from others
      • they must be supported in this ongoing learning at the same time they are held accountable for sustaining their collective inquiry until there is tangible evidence of improved results
    • How do teachers develop the assessment literacy that can serve as a catalyst for improved pedagogical practice?  Again, work as collaborative teams to build common assessments instead of working as individuals.
      • Collaborative teaming is considered essential to building the assessment literacy of educators (emphasis mine)
    • best way to deliver solid PD to teachers – ensure that the school is operating as a professional learning community (PLC)
    • Therefore, school leaders must:
      • establish moral imperative of helping all students (emphasis mine)
      • assigning teachers into meaningful teams that share a collective responsibility for helping students (emphasis mine)
      • providing time for teachers to work together on curriculum, instruction, and assessment as part of their routine practice

Interleaving and spacing practice

How might we help students with learning and remembering without giving them 30 of the same math exercises each night? 

This is a question that I had pondered for a while.  I really enjoy reading about cognition, so in addition to my cognition book for grad school, I was also reading “Why Don’t Students Like School?” by Daniel Willingham (@DTWillingham) and “Make It Stick” by Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel.  The topics of interleaving and spacing practice kept coming up.  While it is more difficult for learners, it helps increase the “stickiness” of what they are learning.  Willingham states, “But something else does protect against forgetting: continued practice” (p. 117).

Interleaving is the opposite of how most math teachers assign practice work.  Typically, math teachers assign massed practice – students work out examples that are all on the same topic.  Interleaving is mixing up the topics. This is much harder and slower for learners initially.  From Make It Stick, “…research shows unequivocally that mastery and long-term retention are much better if you interleave practice than if you mass it” (p. 50).

Spacing is what it sounds like, spreading out practice instead of cramming.  If you will study for 3 hours, it’s better to space out that 3 hours instead of doing it all at once the night before a test.  You will forget less and remember longer by spacing.  More from Willingham, “If, on the other hand, you study in several sessions with delays between them, you may not do quite as well on the immediate test but, unlike the crammer, you’ll remember the material longer of the test” (p. 119).  Spacing is for the long term.  As a teacher, I want my students to remember for the long term.  The math they are doing in my class will continue to be built upon in future coursework.

Catalyst for change

Even with reading about this in 3 different books, I still hadn’t made any changes to the practice work I assigned to my students.  Then in November 2014 I attended the NCSM Regional Conference in Richmond, VA and heard Steve Leinwand (@steve_leinwand) speak for the first time.  (If you’ve never heard him live, I highly recommend rectifying that situation!)  Steve also spoke about spacing, interleaving, and giving students no more than 8 practice problems per night.  That was it, I was sold.  I couldn’t escape that I was being directed to change my assignments.  I try to have one major takeaway from any conference that I implement immediately – changing how I did practice was done my first day back at school after the conference.  I told students what I was doing and why.

Here is how it looks (typically) in my classroom.

New Unit:

Day 1 – 8 problems – 4 are low-level on the new material, 2 are medium-level on review material, 2 are high-level on review material

Day 2 – 8 problems – 4 are low-level on the new material, 2 are medium-level on Day 1 material, 2 are high-level on review material

Day 3 – 8 problems – 4 are low-level on the new material, 2 are medium-level on Day 2 material, 2 are high-level on Day 1 material

This pattern repeats throughout the unit.  It cuts down on end of unit review time because we’ve been reviewing all unit long.  Additionally, we use ALEKS & I would assign some exercises on ALEKS that were review.  I could see details of each student with each topic and use the weak topics as our warm up the next day in class.  New this year: our math team has agreed to have all unit tests include some amount of material from previous units.  We are hoping this shows students the importance of remembering what they’ve learned.

If you want to read more on this, I recommend the 3 books listed at the top of this blog post.  In addition, read anything and everything by the incomparable Henri Picciotto (@hpicciotto). He has an excellent post on how he lags practice.  I haven’t done lagging that way, yet!  Finally, you may want to check out the #eduread thread on Twitter.  A conversation I had there last week inspired this blog post!

#EdCampATL session on Hattie

Here are the links to Hattie resources I’m using today:

The Keynote I was planning to use! ha!

1. “Quiz” on what has the highest effect size:

2. Answers to the “quiz” (don’t peek! take the quiz in #1 before you check your answers!):

3. Full list of Hattie Effect Sizes:

Links to books I mentioned:

Hattie’s “Visible Learning”

Hattie’s “Visible Learning for Teachers”

Hattie’s “Visible Learning & the Science of How we Learn”  <—-haven’t read this one yet

Daniel Willingham’s “Why Don’t Students Like School”



Teaching Methods & How Students Learn – my educational views

Edit 8/29/13:  This post was originally titled about my reflections from the NCTM Interactive Institute in DC earlier this month.  However, the post ended up not reflecting on the actual sessions at all, but became my treatise on how I view education.  Therefore, I changed the name of the post.  The content below is the same.

This was my second NCTM event, my first being the national convention back in April.  I must say, the one thing that made this better is not arriving in a snow storm! However, I missed Desmos, Mathalicious, & math trivia.  This blog post will start with my reflections on the theme of the conference and then I will give highlights of the sessions I attended.

As I left the conference yesterday, I had lots of thoughts swirling about.  Cathy Seeley closed with a recap of the 2.5 days plus some added commentary.  As I sat there, I wondered if others felt as inadequate, and maybe even confused, as I did.

The focus of the Institute was the CCSS Standards for Mathematical Practices.  We spent a lot of time on rich tasks.  Excellent tasks, some that would be too far over the head of my students, but excellent tasks nonetheless.  Here is my struggle.  Are they saying that every single thing we teach should be done through a rich task?  I know that we “cover” too much in one year in American mathematics. I know that deeper is better than broader.  I also know if I teach everything through tasks, we won’t even get close to what needs to be accomplished in a year.  Things I’m held accountable for.  I should also add that I tend to be a cognitivist in my educational thinking more than a behaviorist or constructivist.  I believe all 3 are needed in education, but those are my leanings.

The other struggle I’m having relates to various analyses of research that I’ve read (or am in the process of reading):

Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching by Kirschner, Sweller, and Clark  (I’ll call it KSC below)

Visible Learning, a Synthesis of over 800 Meta-Analyses Relating to Research by John Hattie

Visible Learning for Teachers by John Hattie

For example, my reading of the analysis by KSC leads me to believe that below-level and on-level students benefit most from direct instruction before discovery activities.  In addition, that really only the honors/AP type students would benefit from discovery before direct instruction, and it appears only minimally.  I don’t think anyone would argue we about providing rich tasks for our students.  It seems to me that the discussion is WHERE in the sequencing they should occur and should they be the ONLY types of things we ask our students to do.  From KSC, these rich tasks should come after some direct instruction, not before, and definitely not the only type of instruction.

In addition, both works by Hattie promote the use of direct instruction.  If you don’t have these books and you are a teacher or administrator, you need them. period. Hattie measured effect size on learning of various education related items – contributions from student, home, school, teacher, curricula, and teaching approaches.  He says that most anything you do to increase learning will work to some degree, it’s obviously better than nothing.  However, some things have a better than average rate of increasing student learning.  He found the average effect size to be 0.40. Therefore, if you want greater than average learning for your students, seek to implement the items greater than 0.40.  He also doesn’t say not to do anything lower than 0.40, but those shouldn’t be the only things you do.

Direct instruction has an effect size of 0.59.  Yeah, you read that correctly.  This thing that gets railed against all the time has a proven effect size – kids learn from it.   In Appendix B of the first of his books I mentioned above, it is listed as #26 out of 138 items. It’s effect size of 0.59 is tied with Cooperative vs. Individualistic Learning and Study Skills.  Group work is constantly encouraged, but direct instruction tends to get represented as bad.  Inquiry-based teaching has an effect size of 0.33, lower than average.  Now it’s not bad, it’s just not as high as direct instruction.

Problem-based learning has an effect size of 0.15.  From Visible Learning, pg. 211:
“As will be seen, this is a topic where it is important to separate the effects on surface and deep knowledge and understanding.  For surface knowledge, problem-based learning can have limited and even negative effects, whereas for deeper learning, when students already have the surface level knowledge, problem-based learning can have positive effects.  This should not be surprising, as problem-based learning places more emphasis on meaning and understanding than on reproduction, acquisition, or surface level knowledge.”

Again, it appears to me, that some type of direct instruction for surface learning should come before rich tasks and problem-based learning.

So, when I attend something like NCTM’s Interactive Institutes or read awesome math blogs & tweets, I’m left wondering if I’m doing it all wrong.  I keep hearing and being told that effective math instruction needs to be inquiry and problem based, that direct instruction is bad, and heaven forbid if you give your students a worksheet (though appropriately titled as a graphic organizer).  Then I come back to what I’ve read in these meta-analyses of research and wonder why no one seems to talk about what is being shown.  Am I reading bad research?  I don’t think so, but maybe I’m wrong.  Why can’t we talk in the math world about the benefits of a variety of teaching methods? And let’s be honest, the reality is kids need to just know and memorize some basic math facts and formulas.  The high school student who barely has to think about 8 times 7 being 56 is ahead of the student who needs to reason it out or use a calculator.  This low-level knowledge & recall will help them apply their higher-order thinking skills to deep problems.  When I think of Vygotsky’s ZPD, I don’t think many of my students would be able to do the higher-order stuff without the lower-order stuff at least having been referenced.

I’ve decided that when I attend these types of things or read awesome things on the web, I need to remember “all things in moderation.”  Direct Instruction – yes.  Rich Tasks – yes.  PBL – yes.  Basic Memorization – yes.  Graphic Organizers/worksheets – yes.  Manipulatives/hands-on – yes.  Flipped Classroom – yes.  SBG -yes. Transdisciplinary Ed. – yes.  All of these together (plus more not listed) can create a rich learning experience for a student.

This is long enough, I’ll put my recaps of the sessions in another blog! ha! Any feedback or pushback to my thoughts above is much appreciated!