A Statistical Analysis of the Effectiveness of the Rhoades to Reading Intervention Project
at
Sequoia Middle School
Dr. David V Chavez
California State University, San Bernardino
Suzanne L Reid.
Loma Linda University
Muriel Lopez
Claremont Graduate School
Presented to Jacquie Rhoades & Dr. Maria Palacio
Sequoia Middle School
April 16, 1999
The
purpose of this analysis is to statistically support the effectiveness
of the reading program currently in place at Sequoia Middle School in
the Fontana Unified School District using reading scores previously
collected in 1997 & 1998. It is hypothesized that those students
who participated in the reading intervention program would show greater
growth between 1997 & 1998 than those students that did not
participate. Towards this end, two hundred and thirty-one children who
participated in the intervention and had both 1997
& 1998 reading scores were compared to one hundred and four
children who did not participate and had both 1997 & 1998 reading
scores (non-intervention group). Children were selected to participate
in the intervention based on initial assessment of reading ability with
the poorer readers entered into the intervention group. The mean score
for the children in the intervention group on the 1997 reading test was
4.197. The mean score for the children in the non-intervention group on
the 1997 reading test was 4.483. A t-test revealed that this difference
approached significance (t=1.97, df=333, p=.063). While not
significant, the difference in the initial mean scores strongly
recommended a second analysis that will be described later. A second
t-test was run for grade equivalent growth of each group. This t-test
revealed that the mean gain of .977 years for the intervention group
was significantly greater than the mean gain of .507 years for the
non-intervention group (t=3.49, df=207, p=.001). Interestingly, a third
t-test revealed that the 1998 grade equivalent mean reading scores of
4.894 for the intervention group and 4.989 for the non-intervention
group were not significantly different from each other.
Taken
as a set, these analyses suggest that the two groups tested at
relatively different levels in 1997, with those children placed into
the intervention scoring below those that were not placed in the
program. Test scores for 1998 revealed that in the intervention program
gained almost a full grade in their abilities, while those that did not
participate gained about half a grade. This difference in growth in a
meaningful one as exhibited by the fact both groups’ reading skills are
comparable in 1998.
As
mentioned earlier, the groups were not equal in their pre-test reading
scores. This difference can be perceived as a limitation of the
previous analysis. It can be argued that a group that begins at a lower
level stands to gain more ground in the following year. While this
argument belies the reality seen in education that low achieving
children fall further and further behind year rather naturalistically
“catching up”, it is a legitimate research critique. Therefore, a
second analysis of selected cases was conducted.
In
this second analysis, we selected out cases from the larger group – the
intervention group – so that the remaining group would be comparable to
the non-intervention group. In order to do this, the distribution of
scores for both groups was remarkably similar (1.5 – 7.9 for the
intervention group and 1.5 – 9.9 for the non-intervention group). It
was the frequency distribution that was different with greater number
of lower cases in the intervention group. This is consistent with the
initial selection process for inclusion in the intervention group.
Cases were randomly selected using a random numbers table for deletion
in those instances where there existed a large number of cases in the
intervention group (primarily in the lower range). A single case was
also left at each score level in the intervention group when there was
no representation in the non-intervention group to maintain some degree
of semblance to the original sample. In addition, a single case was
deleted from the non-intervention group because the reading score of
9.9 had no comparable matched case in the intervention group. This left
a sample of 111 in the intervention group with a range of scores
between 1.5 and 7.9 and a sample of 103 in the non-intervention group
with a range of scores between 1.5 and 8.1.
Once
case selection was done to create matched samples, the same series of
t-tests were conducted. The mean score for the remaining 111 children
in the intervention group on the 1997 reading test was 4.475. The mean
score for the remaining 103 children in the non-intervention group on
the 1997 reading test was 4.430. A t-test revealed that this difference
did not approach significance. This suggests that the groups were now
comparable and the results from this second analysis would not have the
same sample limitations as in the previous analysis. A second t-test
was run for grade equivalent growth of each group. This t-test revealed
that the mean gain of .934 years for the intervention group was
significantly greater than the .554 years for the non-intervention
group (t=2.15, df=212, p=.033). Interestingly, a third t-test revealed
that the grade equivalent mean scores of the intervention group (5.409)
now significantly exceeded that of the non-intervention group (4.984)
(t=2.13, df=212, p=.034).
Taken
as a set, these analyses suggest that the random selection of cases was
successful in creating a comparable subset of intervention group scores
for analysis. Test scores for 1998 revealed that those children in the
intervention group gained almost a full grade in their abilities while
those that were not in the intervention gained about half a grade. This
difference in growth leads to the subset of comparable intervention
children actually exceeding the reading abilities of the
non-intervention children in 1998.
These
analyses conducted suggest that the reading intervention was a
successful one. Children are making meaningful gains that are helping
them reach and exceed the reading level of children who are selected
due to their higher initial reading scores. While this support is
compelling, further research might randomly assign comparable cases
from the outset to the two groups to determine whether the same
findings persist.