On Oct 15, the Arizona Department of Education released it first school labels under the year-old Arizona LEARNS program. Every school in the state received one of the following markers: underperforming, maintaining performance, improving or excelling.
To Sheri Wenzel, parent of children in third and seventh grade at Coronado K-8 school, the label of "maintaining performance" that her school received did nothing but confuse her because she'd also been told that her school was "underperforming" according to federal Title I guidelines.
"I think the labels have confused more people than helped them," Wenzel said. "First we get a label saying we're one thing, then another label saying something different. And the funny thing is, before the news came out about these labels, I was talking with parents at a soccer game and we were all talking about how we feel confident in this school. I've been so impressed by the staff and the administration and then, Boom! This comes out."
For the average person, the push for school accountability and the programs associated with it - Arizona LEARNS, No Child Left Behind and Title I - can seem a convoluted mess. Add in the A+ School and Blue Ribbon programs and a quagmire develops that could sink most folks in a hurry.
Although all the programs mentioned have to do with rating schools, they are not all tied together, said Tara Teichgraeber, public information officer for the Arizona Department of Education.
"A+ and Blue Ribbon are private programs that have nothing to do with state or federal guidelines," she said. "And there are not really two systems - one federal and one state - even though that is what people have thought with No Child Left Behind. The way to look at this is that the federal government has created guidelines for states to use to shape their own (accountability and rating) programs. No Child Left Behind is direction from the federal government for states to create their own programs."
On Jan. 3, President George W. Bush signed into law the No Child Left Behind Act. He developed the program "to close the achievement gap with accountability, flexibility and choice, so that no child is left behind," according to the text of a speech he gave when signing the bill that was posted on the NCLB Web site.
The act emphasizes school improvement through early literacy, scientifically based instruction methods, more detailed and extensive teacher preparation and - perhaps the most controversial component - parental school choice. The law requires all states to develop state curriculum standards, a method for measuring those standards and a method for ascertaining average yearly progress for all public schools.
Arizona adopted state standards in 1996 and a method for testing those standards - the Arizona Instrument to Measure Standards, or AIMS - was first administered in 1999, said David Garcia, assistant state superintendent for standards and accountability. The state academic standards describe what students must know and be able to do as they move from kindergarten through high school in nine different subject areas.
"It is not just understanding, but performing as well," Garcia said. A detailed list of standards can be found at www.ade.az.gov/azlearns.
The standards were developed due to increasing public pressure for school accountability under the Clinton administration, Teichgraeber said.
"Arizona was really just ahead of the ball in getting standards (before required by NCLB)," she said.
Arizona's method for ascertaining progress as required by NCLB is Arizona LEARNS, said Teichgraeber, and that program was developed in response to Proposition 301, which voters passed in November 2000 to provide more resources to schools in exchange for greater accountability.
Before NCLB, the only schools subjected to any labels were those in the federal Title I program.
In 1965, the federal government established Title I as part of the Federal Elementary and Secondary Education Act. Title I provides school districts extra financial resources to help improve education in high-poverty schools and equalize educational opportunities to ensure that poor and minority students have the same learning opportunities as children in more affluent neighborhoods.
In order for schools to receive federal Title I monies, a minimum of 40 percent of the student body in a school must qualify for the federal free lunch program. To qualify, a student must demonstrate financial need which is determined by the number of people in his or her family and the family's gross annual income. A family of four can make no more than $23,530 to qualify for the free lunch program, according to Debbie Boggs, bookkeeper for Amphitheater Public Schools food service department.
To continue to receive Title I funds, a school must demonstrate Adequate Yearly Progress as shown by scores on the Stanford Achievement Test, ninth edition, (commonly referred to as the Stanford 9), or, in the case of Arizona schools for the past three years, how the school did on the AIMS test. The only labels used in the Title I program are "in need of improvement" (sometimes called underperforming), or "not in need of improvement," according to Teichgraeber.
"The federal government had a system to improve Title I schools and that used labels, and now we are transitioning everyone into one system (under No Child Left Behind guidelines) and there is some confusion" over label discrepancies, Teichgraeber said.
The confusion Teichgraeber recognizes and Coronado parent Wenzel bemoans come primarily from the overlap of the soon-to-be-defunct Title I descriptions and Arizona LEARNS.
Under Arizona LEARNS, there are no underperforming schools in Amphi because all schools showed Adequate Yearly Progress as measured by LEARNS criteria, said Julie Thayer, director of Title I programs for Amphi. There are, however, four schools - Holaway Elementary, Amphi High, Nash Elementary and the kindergarten through fifth-grade section of Coronado K-8 - that are labeled by Title I guidelines as being "in need of improvement" or underperforming.
When a Title I school is labeled underperforming for two consecutive years, it is placed in a program called "school improvement," said Thayer. Until NCLB became law, a school needed to show Adequate Yearly Progress for two out of three years to move out of school improvement. Under the new guidelines, that school must show that progress for two consecutive years.
To illustrate the problem of a school suffering under two conflicting labels, Thayer explained that if a Title I school did not meet Adequate Yearly Progress objectives in the 1997-1998 and 1998-1999 school years, that school was placed in school improvement. In order for the school to come out of school improvement, there would have to be measurable gains via Arizona LEARNS standards - measured using both AIMS and Stanford 9 scores - for the 1999-2000 and 2000-2001 school year.
However, because the State Board of Education was developing the Arizona LEARNS program during the 2000-2001 school year, that year was considered a "forgiving" year where school ratings were concerned, said Thayer.
Therefore, schools that improved during the past two years are still under school improvement for one more year because "we were in limbo while they were developing Arizona LEARNS," she said, and improvements made during the 2000-2001 year did not count toward emancipation from school improvement "because we did not have a state measure to use yet."
"This (labeling change) may take a couple of years for our Title I schools depending on how long it takes them to make gains," said ADE's Teichgraeber. "A school can be in school improvement for four years, so it depends on where these schools are in that schedule as to if they will be out of school improvement and transitioned over to the new labeling next year or the following one. We really hope with our new school improvement plans in place we can move them out of school improvement more quickly and that this will be better for all our schools. With No Child Left Behind there is only one labeling system - that which each state uses to comply with the federal law. But for a few years, while we still have some Title I schools under the old system, there might be some confusion over labels."
"The labeling system is very complicated and when you are working with two systems, it is more so," said Thayer.
Dave Spire, counselor at Canyon del Oro High School, agrees that the labeling system is confusing, but also is adamant that the labels are not meaningful.
"The problem is that there is no definitive description of what these labels are and it is difficult for anyone to know exactly what they mean," Spire said. "Labels are not helpful in and of themselves. What would be more helpful would be descriptions of results and programs in schools that could be improved for students to achieve at a higher level."
In addition, Spire said, there are assumptions being made about the tests used to come up with the labels, particularly the AIMS.
"The first is if AIMS is a valid and reliable test in regard to measuring where students are. And two, is the AIMS test meaningful? … These seem to be arbitrary standards that have been established by someone, somewhere, in order to measure how well schools and students are performing without any direct correlation to day-to-day life," he said. "It is difficult for me to understand how labels relate to basic information students need to be successful after high school."
Consequences for schools that fall into the underperforming category of Arizona LEARNS vary depending on how long the school fails to show progress, said Teichgraeber.
If a school falls into the Arizona LEARNS underperforming category, the state - under direction from NCLB - requires the school to develop improvement plans incorporating strategies from scientifically based research and present them to the state and school community. If the school receives an underperforming ranking the next year, it is said to be "failing" and it must notify the school, parents and local community of its status, allow students to transfer to another public school if they want to, provide tutoring and support services and be willing to accept state intervention in terms of possible school reorganization, Teichgraeber said. If a school remains failing the next year, it is subject to complete takeover by the state or a private organization. In addition, the Arizona LEARNS statute states that if "more than one-half or at least five schools in one district are failing," then upon the next election of the local school governing board the ballot will contain a statement recognizing that so many schools are failing," Teichgraeber said.
Another glitch in the labeling confusion surrounds the A+ and Blue Ribbon school awards. The A+ awards are granted by the Arizona Education Foundation, a nonprofit organization that "works to identify excellence in education and recognize and reward it," said Bobbie O'Boyle, executive director of the foundation. In addition to the A+ program, the foundation sponsors the state spelling bee and the Arizona Teacher of the Year program.
A school can receive an A+ award by demonstrating academic excellence through a number of factors including student support, school culture, unique and challenging curriculum, school family and community partnership and test scores.
"The A+ system is a more comprehensive labeling system than those that rely simply on test scores," said O'Boyle.
Until the establishment of NCLB, an A+ school could apply to become a national Blue Ribbon School, O'Boyle said.
"They got rid of the Blue Ribbon program this year and No Child Left Behind does have a Blue Ribbon section, but it is based primarily on test scores now, so you do not get to identify good schools on a more broad range," she explained.
Arizona has been granted permission from the federal government to use the Arizona LEARNS system until next year, said Garcia, and is currently developing a proposal to pitch to the U.S. Department of Education in the spring for Arizona LEARNS to be permanent.
Garcia said NCLB, as written, is considered more stringent because it only looks at results from state tests tied directly to curriculum; in Arizona's case, the AIMS test. Arizona is proposing the federal government accept LEARNS measurements in place of NCLB "because when you include both AIMS and MAP scores you get a more comprehensive look at progress," Garcia said. (See stories pages 19 and 20 for description of AIMS and MAP).
"NCLB is more stringent, but less accurate," he said. "The status model the feds use only looks at one grade every year - third grade for example. When you test a cross section you are looking at kids' progress as they are going through school - you look at the third grade, then see how that same third grade did in fourth grade. We will be making the argument that we are following the (NCLB) law, but that we want more included. It makes sense and we think they will see that."
Teichgraeber said North Carolina and Louisiana are also lobbying the Department of Education to include longitudinal measures in addition to the cross section model provided by such tests as AIMS.
There are no federal consequences to states for having failing or underperforming schools, Garcia said; the federal consequences are tied to states not developing standards and assessments, both of which Arizona has. Should a state not have those standards and assessments, the U.S. Secretary of Education is allowed to withhold an unspecified amount of federal funds from that state, said Garcia.
Arizona received $173 million for Title I schools last year and $314 million in federal grants for other programs such as teacher training, assistance to immigrants, after school programs and rural school aide, said Teichgraeber. Locally, the Amphi district received approximately $5.5 million in federal grant monies, of which approximately $2.3 million was designated for Title I, said Ron Pierce, grants accountant for Amphitheater Public Schools.
School labels will come out each year in August, said Garcia, and eventually there will be only one set, ending the confusion that happened this fall. If the Arizona Department of Education can fully convince the federal government of the worthiness of using the Arizona LEARNS method to designate school profiles, that is the standard that will be used. If not, the state will either use the NCLB labels or develop another program that comes into compliance with NCLB, Garcia said.
"But I'm very confident they will accept the Arizona LEARNS model because this is a more comprehensive measure," he said.
In the end, what do the labels actually mean in terms of Johnny learning how to read?
"It is definitely making schools accountable," said Harelson Elemen-tary School Principal Andrew Heinemann, "but I wouldn't say labeling is helpful in and of itself. It can be very confusing to parents and teachers because the method (to arrive at the labels) is complicated. But our goal is to have all students reach one year's growth in each subject each year. So we are looking at specific data (from the tests) to help us reach the goals and improve the quality of teaching."
Heinemann said because this is the first year of NCLB and Arizona LEARNS that "only time will tell" if the labels will be helpful in the long run.
"I don't know what will happen in future years. I think it will be interesting to see how schools with relatively high test scores progress through the labeling process over time," he said. "But right now, the date is helping us drive our instruction and decision-making. I think we need to work hard to properly educate parents on the process and let them know all the good things going on in schools."
ARIZONA LEARNS LABELS
The following is a description of the Arizona LEARNS Achievement Profiles, also referred to as labels, from the State Department of Education.
Underperforming: needs to meet state performance and
progress goals. School performance was below the state base
line in 2000 and did not make adequate progress OR the
school started above the state baseline and did not make
adequate progress through 2002.
Maintaining performance: meets state performance goals,
but needs to meet state progress goals. School performance
remained on par through 2002.
Improving: exceeds state performance and state progress
goals. School performance was above or below state baseline
in 2000 and the school surpasses expectations through 2002.
Excelling: More than 90 percent of school's students meet
or exceed state standards consistently from 2000 through
2002 on AIMS and 90 percent demonstrate appropriate One
Year's Growth; and, if a high school, 90 percent graduate in
five years, and less than 6 percent drop out each year.
The schools in the Amphitheater Public Schools system that were labeled as maintaining performance on Oct. 15 are:
Amphitheater High School, 125 W. Yavapai Road
Coronado K-8 School, 3401 E. Wilds Road
Nash Elementary School, 515 W. Kelso St.
Keeling Elementary School, 2837 N. Los Altos
Prince Elementary School, 125 E. Prince Road
La Cima Middle School, 5600 N. La Canada Drive
Cross Middle School, 1000 W. Chapala Drive
Rio Vista Elementary School, 1351 E. Limberlost Drive
The schools labeled improving are:
Harelson Elementary School, 826 W. Chapala Drive
Wilson K-8 School, 2300 W. Glover Road
Mesa Verde Elementary School, 1661 W. Sage St.
Donaldson Elementary School, 2040 W. Omar Drive
Lulu Walker School, 1750 W. Roller Coaster Road
Holaway Elementary, 3500 N. Cherry Ave.
Copper Creek Elementary School, 11620 N. Copper Spring Trail
Canyon Del Oro High School, 25 W. Calle Concordia
Amphitheater Middle School, 315 E. Prince Road
There were no schools labeled excelling or underperfoming in the district. Ironwood Ridge High School received no label because it was not open during 2000, the baseline year for test scores.
STANDARDIZED TESTS: STANFORD 9 VS. AIMS
The Arizona Department of Education compiles school achievement profiles by using the Stanford 9 and AIMS tests for elementary and middle schools. It uses a 3-year rolling average of scores that captures performance trends benchmarked against state performance scores during the years 2000 through 2002. In addition, for high schools, the five-year graduation rate and the annual dropout rate are factored into the label received.
The Stanford 9 compares Arizona students with their peers nationally and is given to students in grades 1 through 9. The Measure of Academic Progress (MAP) is a summary based on the Stanford 9 scores that measures a student's progress from year to year and is the measurement used by Arizona to help calculate where a school falls on the Arizona LEARNS scale of underperforming, maintaining performance, improving or excelling.
In general, the subject matter on the Stanford 9 is not tied to school curriculum, which appears to be at least part of the reason No Child Left Behind regulations do not wholeheartedly endorse this method of testing for school profiles. Stanford 9 results are reported in terms of a national percentile rank. A child who places in the 75th percentile is doing better than 75 percent of the students his age across the nation.
Because the Stanford 9 is not tied directly to curriculum - and because the United States does not have a national curriculum - a child might be getting 90 percent in mathematics classroom work at his school but, when taking the Stanford 9, still place only in the 75th percentile because the questions on the Stanford 9 might cover work he'd not yet been exposed to at his particular school.
On the other hand, the AIMS test is tied directly to curriculum and is administered to third, fifth, eighth and 10th graders annually. When students are tested, they are given reports based on their success using the terms "falls far below standards," "approaches standards," or "meets or exceeds standards." The AIMS test is similar to other state's standards-based accountability tests and has been somewhat controversial among parents and teachers. Critics assert that having curriculum tied directly to a state test and then the school's performance on that test tied directly to a school's label -- and often to that school's teacher-incentive program - results in teachers "teaching the test" instead of engaging in more creative teaching endeavors.
"It is an absolute fact that we have been pressured since AIMS started to teach to the test," said Marc Sabb, physics and astronomy teacher at Canyon del Oro High School. "Our teachers' performance pay is directly tied to how well our students do on the AIMS and our administrators' jobs are directly linked to how well the students perform on the test. So by virtue of those things alone, we are forced to teach to the test rather than teach the curriculum that we, as professionals, deem is best for our students."
Rhonda Ball, a first-grade teacher at Mesa Verde Elementary School, said teachers at her school view the AIMS "as a reliable, valuable instrument," but that it "limits their ability to teach special lessons or do added creative learning experiences they might have before AIMS because of the time restraints of teaching all the standards."
"For example, the elections that just happened - what a wonderful learning experience that would have been. But there just wasn't enough time for most of the teachers who have to deal with (preparing for) AIMS," Ball said.
Beginning in 2006, passing AIMS will be a high school graduation requirement; students who do not "meet or exceed" the state standards in mathematics, reading and writing will not be able to graduate.
For a school to be labeled as maintaining performance under Arizona LEARNS, it must receive 15 to 35 points combined from a nine subject-grade formula. A K-8 school would examine both the Stanford 9 and AIMS scores of third, fifth and eighth graders in reading, writing and math. (Three subjects multiplied by three grades results in the nine subject-grade formula).
Points for individual grades are awarded based on two things, said Julie Thayer, Title I director for Amphitheater Public Schools. The first is the three-year average growth in percentage points on the Stanford 9 (reported in terms of MAP scores) and the second is a three-year average of students in a particular grade and school who move out of the "falls far below" category in the AIMS into at least the "approaches" category, and the percentage of students who move out of the "approaches" category into "meets or exceeds."
This year, the baseline for a school and grade is the percentage of students in the meets or exceeds category of the AIMS in the year 2000. For instance, at Coronado K-8 School, 3401 E. Wilds Road, 63 percent of that school's third graders met or exceeded state standards in math in 2000. Using that baseline, the school then charts the three-year average of the percent of students moving out of the falls far below into approaches and the percent of students moving out of approaches into meets or exceeds. Since the year 2000, third-grade at Coronado has had 2.98 percent of students move from approaches into meets or exceeds. However, it had a slight increase of students in the falls far below category, 0.59 percent. To figure out its "growth points," the 0.59 added to the falls far below category must be subtracted from the 2.98 moving into the meets or exceeds category for an AIMS growth point total of 2.39.
In addition, the school looks at the three-year average in Stanford 9 scores as tracked through the MAP. For this same Coronado class, that number was 72 percent of students making one year's growth on the Stanford 9. The growth points awarded for that percentage is 2.1. Therefore, Coronado's third grade math class received 4.49 growth points (2.39 plus 2.1) which, when charted against the number of students in the meets or exceeds baseline year, gives that grade in that subject at that particular school a score of 3 (see chart, page 20).
This complicated formula is followed for every grade tested in each of the three subjects at every school. When all those growth points are added together, the resultant number is what determines a school's label. Coronado's combined nine subject-grade number was 31, five points below the cutoff of 36 points needed to get into the "improving" category of Arizona LEARNS.
Wilson K-8 School, 2300 N. Glover Road, was labeled "improving" under Arizona LEARNS because their compiled nine subject-grade formula was 49 and to be labeled improving a school's score must be between 36 and 62. Wilson's third graders did the same in their math growth point score as Coronado did, but in reading and writing at that grade level, Wilson received scores of 7 while Coronado received scores of 3.
Wilson has 12 percent of its student body on free or reduced lunch through the Title I program; Coronado has 77 percent of its student body qualifying.