Guiding Questions for MSL Systems:
Overarching System Design

Balanced Assessment System
Decisions about which assessments and measures to use and value within a district are best made on the basis of district goals and priorities and the expectations for what students need to know and be able to do as outlined in the Colorado Academic Standards. MSL systems that are founded directly on a district’s overarching assessment framework connect evaluation measures to the priorities of the district and are better able to motivate instructional changes and give educators meaningful information about teaching and learning. CDE’s Measures of Student Learning guidance includes a six-step process that helps ensure the measures selected as part of an evaluation system thoughtfully reflect district values.

How has your district grounded your MSL system in the Colorado Academic Standards?

  • How have you ensured that the Colorado Academic Standards are the driving force in determining which assessments are important to your teachers and district leaders?
  • Does your district’s assessment system reflect the expectations for what students should know and be able to do as outlined in the Colorado Academic Standards? Was it created by first critically examining your instructional priorities and aligning assessment systems to the most important elements? Does it include a variety of summative, interim, and formative assessments at each grade level and content area?
  • What evidence do you have that the assessments included in each educator’s evaluation reflect your district’s priorities and the range and scope of key activities undertaken in the classroom?

How has your district used your comprehensive assessment system to design an MSL system that reflects district goals and priorities?

  • Is your district’s assessment system balanced? Does it reflect your district’s goals and priorities?
    • How have you ensured that the MSL system aligned with your district’s broader assessment system, but is not singularly driving decisions about what is considered important to your teachers and district leaders?
    • What evidence do you have to suggest the assessments used in your MSLs provide a comprehensive picture of student performance for each educator and give teachers meaningful information about student learning?
    • How is each assessment related to the relevant grade-level and content standards and adopted curriculum?
    • In what ways do the collection of assessments used in your system work together to provide a holistic picture of student learning?
    • Is the assessment schedule mapped meaningfully to the cycle of instruction for each grade/subject?
  • Have you ensured that each educators’ measures of student learning are:
    • Aligned to your district’s values?
    • Aligned to the priorities and needs set forth in your district’s Unified Improvement Plan?
    • Reflective of the unique context of your district and students?
  • Have you been thoughtful in using assessments for their intended purposes?
MSL Comparability and Fairness
The State Board of Education adopted rules for S.B. 10-191 that require measures used as part of an MSL system to be comparable among teachers of similar content areas and grades. This focus on comparability is an important step in ensuring fairness for educators — defined loosely as a system where all educators from various content areas and grade levels, serving various student populations, have an equal opportunity to earn each rating.

What evidence do you have that it is equally challenging for all teachers — regardless of content or grade level — to earn each rating?

  • Is the proportion of collective versus individual attribution relatively consistent across educator types? If not, what rationale does your district provide for differences in collective and individual measures across different types of educators?
  • Current-year assessment data can only be used in MSLs if results are available at least two weeks prior to the last day of school. This limits the use of most state assessment data to data from the prior school year (as well as some other assessment results that may not be available in time). Have you thoughtfully considered how this impacts measure selection with regard to fairness and comparability?
    • How does this impact the selection of measures for new educators in particular, for whom assessments from the prior year are not available?
Unintended Consequences
Decisions regarding the inclusion (or exclusion) of measures in an MSL system come with a host of intended and unintended consequences, which may impact student learning, educator satisfaction, and school culture/climate. In reflecting on your MSL system, consider whether it creates unintended consequences.

How has your district considered potential unintended consequences with regard to both system design and communication with stakeholders?

  • What does your system encourage teachers and/or principals to do or focus on?
    • In what ways might your system inadvertently prioritize certain content or curriculum over others
    • How might the measures used in your system encourage or discourage educator retention or recruitment in certain grades or subjects, or certain schools or programs (e.g., by encouraging a belief that it may be easier for some schools or subjects to earn an effective rating)?
    • Are there any other ways in which your system might send teachers or principals the wrong messages, or encourage them to do things that you do not necessarily want them to do (e.g., by incentivizing teachers to “teach-to-the-test” by focusing on certain content over the full curriculum because of how that learning will be reflected on assessments used in their evaluations)?
  • Have you considered the unintended consequences that may be associated with the selection of growth [e.g., Student Growth Percentiles (SGPs)] versus achievement (e.g., percent proficient) measures? (See the State Summative Assessments section for more information.)
  • What are your district’s business rules for student inclusion in measures of student learning (e.g., which students are attributed to which educators)?
    • Are certain student groups disproportionately impacted by these rules?
    • Are there students who are unassigned to any educator? What message might this inadvertently communicate about the value of those students’ learning?
    • Do your inclusion rules provide fair and meaningful sets of data points for all educators?
    • Are your inclusion rules easy for educators to understand?
  • How might your district get feedback on potential unintended consequences from teachers, building leaders, and district leadership?
  • How might your district seek an outside perspective on your system in order to more effectively identify potential unintended consequences?

Facebook Twitter LinkedIn Share on Google+
Back to Top