Rhoades to Reading 1st Edition Research Dr. David Chavez
A Statistical Analysis of the Effectiveness of the Reading Intervention Project at Sequoia Middle
School
David V. Chavez
California State University, San Bernardino
Suzanne L. Reid
Loma Linda University
Muriel Lopez
Claremont Graduate School
Presented to Jackie Rhoades & Dr. Maria Palacio
Sequoia Middle School
Fontana Unified School District
April 16, 1999
The purpose of this analysis is to statistically support the effectiveness of the reading program currently in place at Sequoia Middle School in the Fontana Unified School District using reading scores previously collected in 1997 & 1998. It is hypothesized that those students who participated in the reading intervention program would show greater growth between 1997 & 1998 than those students that did not participate. Towards this end, two hundred and thirty-one children who participated in the intervention and had both 1997 & 1998 reading scores (intervention group) were compared to one hundred and four children who did not participate and had both 1997 & 1998 reading scores (non-intervention group). Children were selected to participate in the intervention based on an initial assessment of reading ability with poorer readers entered into the intervention group. The mean score for the children in the intervention group on the 1997 reading test was 4.197. The mean score for the children in the non-intervention group on the 1997 reading test was 4.483. A t-test revealed that this difference approached significance
(t=-1.97, p=.065). While not significant, the difference in initial mean scores strongly recommended a second analysis that will be described later. A second t-test was run for the grade equivalent growth of each group. This t- test revealed that the mean gain of .977 years for the intervention group was significantly greater than the mean gain of .507 years for the nonintervention group (t=3.49, d1--207, p=.001; see figure 1). Interestingly, a third t-test revealed that the 1998 grade equivalent mean reading scores of 4.894 for the intervention group and 4.989 for the non-intervention group were not significantly different from each other.
Taken as a set, these analyses suggest that the two groups tested at relatively different levels in 1997, with those children placed into the intervention scoring below those that were not placed into the program. Test scores for 1998 revealed that children in the reading intervention
group gained almost a full grade in their abilities, while those that did not participate gained about a half a grade. This difference in growth is a meaningful one as exhibited by the fact that both groups' reading scores are comparable in 1998.
As mentioned earlier, the groups were not equal in their pre-test reading scores. This difference can be perceived as a limitation of the previous analysis. It can be argued that a group that begins at a lower level stands to gain more ground in the following year. While this argument belies the reality seen in education that low achieving children fall further and further behind each year rather than naturalistically "catching up", it is a legitimate research critique. Therefore, a second analysis of selected cases was conducted.
In this second analysis, we selected out cases from the larger group - the intervention group - so that the remaining group would be comparable to the non-intervention group. In order to do this, the distribution of scores for the non-intervention group served as a model for case selection from the intervention group. Interestingly, the range of scores for both groups was remarkably similar (1.5 - 7.9 for the intervention group and 1.5 - 9.9 for the non-intervention group). It was the frequency distribution that was different with greater number of lower cases in the intervention group. This is consistent with the initial selection process for inclusion in the intervention group. Cases were randomly selected using a random numbers table for deletion in those instances where there existed a larger number of cases in the intervention group (primarily in the lower range). A single case was also left at each score level in the intervention group when there was no representation in the non-intervention group in order to maintain some degree of semblance to the original sample. In addition a single case was deleted from the non-intervention group because the reading score of 9.9 had no comparable matched case in the intervention group. This left a sample of 111 students in the intervention group with a range of scores
between 1.5 and 7.9 and a sample of 103 in the non-intervention group with a range of scores between 1.5 and 8.1.
Once case selection was done to create matched samples, the same series oft-tests were conducted. The mean score for the remaining 111 children in the intervention group on the 1997 reading test was 4.475. The mean score for the remaining 103 children in the non-intervention group on the 1997 reading test was 4.430. A t-test revealed that this difference did not approach significance. This suggests that the groups were now comparable and that results from this
second analysis would not have the same sample limitations as in the previous analysis. A second t-test was run for the grade equivalent growth of each group. This t- test revealed that the mean gain of .934 years for the intervention group was significantly greater than the mean gain of .554
years for the non-intervention group (t=2.15, df=212, p=.033). Interestingly, a third t-test revealed that the grade equivalent mean scores of the intervention group (5.409) now
significantly exceeded that of the non-intervention group (4.984) (t=2.13, df=212, p=.034; see figure 2).
Taken as a set, these analyses suggest that the random selection of cases was successful in creating a comparable subset of intervention group scores for analysis. Test scores for 1998 revealed that those children in the reading intervention group gained almost a full grade in their abilities, while those that were not in the intervention gained about a half a grade. This difference
in growth lead to the subset of comparable intervention children actually exceeding the reading abilities of the non-intervention children in 1998.
The analyses conducted suggest that the reading intervention is a successful one. Children are making meaningful gains that are helping them reach and exceed the reading level of children
who are not selected due to their higher initial reading scores. While this support is compelling,
future research might randomly assign comparable cases from the outset to the two groups to determine whether the same findings persist.