The degree to which a measurement procedure produces the same outcome when the same object, event, or construct is measured under identical conditions is the reliability (of any measurement procedure). Validity refers to the ability of the measurement procedure to accurately measure the construct of interest. In other words, we are measuring what we want to measure. "Validity" and "Reliability" are scientific terms utilized to describe whether measurements are meaningless or worthless. In Chapter 2 of the Fundamentals of Behavioral Statistics, the authors lament "if we cannot show that our measurement was reliable and valid, the data we collect is meaningless."
The 'A' to 'F' School Report Card is an Oklahoma public school grading system, endorsed by our State Legislature, State Superintendent of Schools, and Governor, which they say provides the public with data (to make decisions about how our schools are doing). Most State Senators and State Reps believe the system is based on valid and reliable data, which brands schools with letter grades (A, B, C, D, or F) in order to determine each school's effectiveness. The question now becomes: Does the system measure any school's effectiveness in a valid and reliable manner?
During the first four years of the school A through F letter grade system, education experts across the nation (in states with the A-F system) conducted research - to determine the validity and reliability of such a system. Researchers from many major universities, including O.S.U. and O.U., concluded the letter grade system for assigning an 'A', 'B', 'C', 'D', or 'F' grade to individual "schools" is both invalid and unreliable. Many education experts attempted to answer questions such as "Does the school letter grade system provide a method which gives the public a clear understanding of each school's effectiveness?" In order to determine the answer to the question, correlational studies were conducted which compared existing variables in the new A - F system of measurement. In this way, the valid or invalid and the reliable or unreliable correlational levels could be measured. Variables used for analysis included school funding, student learning capacity, community poverty rates, and school letter grades. Researchers determined that all variables were highly correlated. In other words, as one variable increases, all others included in the analysis either increase or decrease in similar fashion. Variables not included in the research were subjective, such as teacher effectiveness and school effectiveness (How does one measure teacher effectiveness?). Some state lawmakers believe, however, that teacher effectiveness can be measured by student test scores. The analyses of research data indicated that schools with higher funding levels received higher letter grades, schools with higher student learning capacities (as judged by student test scores) received higher letter grades, and schools with higher community poverty rates (as judged by free and reduced student lunch percentages) received lower letter grades ('D' and 'F').
The U.S. Department of Education recognized this deficiency in the school letter grade system, so on November 29,2016, dropped the requirement that states provide a single summative letter grade for each school. Many Oklahoma state lawmakers, as well as State Superintendent Joy Hofmeister, had been saying all along that the school letter grade system was invalid and unreliable because of the single summative letter grade issued to schools. In other words, teacher effectiveness cannot be determined by issuing a single summative school letter grade. One would tend to believe that as soon as the federal government dropped the requirement that schools receive a summative grade, states would also end this requirement - as it was branding all teachers in some schools as failures, when in fact the failure was school funding, poverty, or student learning capacity. The State of Oklahoma forged ahead, however, and is actually doubling down on its criticism of teachers - as Senators and Representatives alike passed the new school letter grade system, which includes the single summative school letter grade! The variable which accounted for less validity (invalid) and less reliability (unreliable) than any other factor... continues unabated.
A quick example of invalid and unreliable teacher labeling: I was the Principal at Duncan Middle School (DMS) from 2001 through 2004. In my opinion, we had some of the best teachers in the state at DMS. Examining the 2016 single summative letter grade for DMS reveals that the school received a 'D', not very good by anyone's standards. I know without a shadow of a doubt that DMS teachers are among the state's best, but they've been labeled as 'poor', by state senators and representatives. Schools and teachers across the state have been mislabeled by those senators and representatives who said, in effect - to heck with research, we must prove how bad Oklahoma teachers really are. Even Joy Hofmeister changed her mind, and now believes (erroneously) that state teachers are only as good as their school's single summative grade - whether it's an A or an F.
Side note: To determine which senators and representatives voted in favor of using the single summative school grade as a method of school and teacher disparagement, go to the Oklahoma Legislature Website, type in HB 1693 under "legislation", and then click "votes". I think all public school supporters will be surprised.
No comments:
Post a Comment