The Institute for Education Sciences funded Frost Science, in partnership with the University of Miami, to develop a comprehensive early childhood science curriculum with assessment tools and a professional development program.
The overall goal of Early Childhood Hands-On Science (ECHOS®) is to investigate science as a domain for enhancing overall school readiness and to demonstrate that very young children can learn fundamental science concepts and the process skills associated with higher-order thinking.
The fully developed preschool science curriculum consists of nine units, each containing four guided week-long science lessons that are introduced in 20-minute segments to small groups of children. A teaching assistant and/or volunteer simultaneously works with the rest of the children using related integration cards in three domains: language/literacy, math and creative arts. The iCards are short activities that are easy to set up.
ECHOS focuses on Life, Earth, and the Physical Sciences. The curriculum combines direct instruction with guided inquiry-based science experiences and exploration. Curriculum units are sequenced to present increasingly more complex science process skills in the categories of observing, describing, categorizing, predicting, experimenting and drawing conclusions.
Quasi-Experimental Study
A quasi-experimental study was conducted in 2010 with 30 Head Start Centers in Miami-Dade County to examine the efficacy of the ECHOS model program. ECHOS professional development and classroom implementation occurred over a seven-month period. In summary, the study found that:
- Teachers who had exposure to ECHOS felt more comfortable teaching science and had more positive attitudes and beliefs toward how science can benefit preschool children.
- Children in ECHOS classrooms obtained significantly higher scores on the direct assessment of preschool science.
- Both teachers and children in ECHOS classrooms exhibited significantly higher levels of science-related behaviors relative to non-ECHOS classrooms.
Results are presented below by measurement instrument.
Preschool Science Classroom Observation Tool (PreSCOT)
Science-related behavior ratings of teachers (t(28) = 0.654, ns) and children as a group (t(28) = 0.343, ns) in ECHOS and non-ECHOS classrooms were not significantly different at the beginning of the year. However, at the end of the year teachers (t(28) = -2.777, p < .01) and children (t(28) = -2.683, p < .01) in ECHOS classrooms exhibited significantly higher levels of science-related behaviors relative to non-ECHOS classrooms.
Direct Assessment of Science
Multi-level analyses were conducted (using HLM6) in order to examine the effects of the ECHOS professional development program on children’s science readiness. Children in ECHOS classrooms obtained significantly higher scores at the end of the year relative to children in non-ECHOS classrooms, after controlling for demographic covariates (age, sex, ethnicity) and pre-test science scores (y01 = 23.92, t = 3.48, p < .001).
Preschool Teacher Attitude and Beliefs about Science (P-TABS)
Relative to the beginning of the year, teachers in ECHOS classrooms felt more comfortable teaching science (t(49) = -2.137, p < .05) and had more positive attitudes and beliefs toward how science can benefit preschool children (t(49) = -2.861, p < .01) by the end of the year. Teachers in non-ECHOS classrooms, however, showed no changes from the beginning to the end of the year.
Randomized Controlled Trial
From 2011 to 2013, the Institute for Education Sciences provided funding for Frost Science, in partnership with the University of Miami, to conduct a two-year full-scale efficacy study of the ECHOS professional development model.
- In the 2011-2012 school year, 91 classrooms (45 ECHOS, 46 Control) with no prior experience implementing ECHOS, participated in the first year of the two-year RCT. Baseline data were collected prior to randomization.
- Classrooms were randomly assigned to a treatment group implementing ECHOS or to a business-as-usual control group.
- Approximately ten children from each classroom were randomly selected and stratified by age and gender, to be assessed on their science skills.
- The program served primarily African-American (48%) and Hispanic (51%) children, ages 3-5.
Year One Findings
Preschool Science Classroom Observation Tool (PreSCOT)
Analyses at the end of the first year showed preliminary evidence for the effectiveness of ECHOS on the quality of teacher’s science instruction and on children’s science skills. A series of repeated-measures analysis of variance (ANOVA) revealed that ECHOS teachers significantly improved their scores in use of instructional strategies associated with higher-order thinking skills, specifically, predicting, investigating and evaluating, across the year. Control group teachers did not.
ECHOS Fidelity Observation Tool
This ten-item instrument administered fall, winter and spring was designed to measure adherence to model (5 items) & quality of implementation (5 items) and using a three-point rubric (0, 1, 2). With respect to fidelity, items were created to measure both fidelity “adherence” and fidelity “quality.” Overall, teachers’ fidelity to ECHOS was high. ECHOS teachers showed significant increases in total fidelity and fidelity quality from winter to spring.
Changes in Fidelity Adherence and Fidelity Quality from Winter to Spring
| Fidelity | Winter Score Average (SD) | Spring Score Average (SD) | Change over time Average (SD) |
| Adherence | 8.29 (1.82) | 8.73 (1.51) | Not significant (F = 2.18, p = .147) |
| Quality | 7.69 (2.34) | 8.64 (2.01) | Significant (F = 7.29, p = .01) |
| Total | 15.98 (3.95) | 17.38 (3.19) | Significant (F = 5.37, p = .025) |
P-TABS (Preschool Teacher Attitudes and Beliefs About Science)
P-TABS was administered to teachers in fall and spring of the academic year. There were three subscales: child benefit, teacher comfort and teacher challenges. ECHOS teachers reported significantly greater challenges in teaching science while control teachers did not. ECHOS teachers also rated science as having more benefit for children and felt more comfortable teaching science.
Teacher Science Knowledge Test
ECHOS teachers significantly improved their scores from fall to spring on a test of general science knowledge developed by the UM research team. However, control teachers did not.
Direct Assessment of Preschool Science
Approximately 900 children (randomly selected to 450 per condition) were assessed on science skills in the fall, winter and spring of the school year.
Multilevel models (time nested in children nested in classrooms) were analyzed to determine if ECHOS was associated with children’s growth in science skills–all children showed significant growth in science skills (p < .001). A trend indicated faster rates of growth for children in ECHOS classrooms (p = .069).
While ECHOS did not predict gains from fall to winter, a trend indicated that ECHOS predicted more gains in science skills from winter to spring compared to control classrooms.
Year Two Findings
Does ECHOS improve Head Start teachers’ science teaching?
Using the Preschool Science Classroom Observation Tool, teachers were observed in the spring of 2011 (prior to random assignment to ECHOS and Control conditions) presenting a science lesson of their choice. The first time point baseline prior to random assignment to Treatment (ECHOS versus Control) shows the equivalence of the two groups (.79 versus .81). Analysis of the four subsequent time points after random assignment and the beginning of the ECHOS program (Winter, 2011, Spring 2012, Fall, 2012, Spring 2013) were conducted using SPSS version 21 with the GLM procedure using a repeated measure design. Treatment (ECHOS versus Control) served as a between subject variable and time as a within subject variable. These data are shown in figure 1 below. ECHOS teachers had a higher mean PreSCOT score across these 4 time periods compared to Control teachers (.85 versus .74). This difference was statistically significant, F(1,64) 4.43, p = .039.
Evaluations of ECHOS Replication – Shared by Organizations
Kellogg ECHOS Professional Development and Family Engagement Program
Year One Evaluation Report 2014-2015
Program Outcomes
Early evaluation results indicate that ECHOS increases children’s knowledge of science concepts. During year one, a pilot study was conducted in which a sample of five children from each classroom (n=165) completed pre and post-tests for each of the six ECHOS curriculum units. The percentage of students who increased their unit assessment score varied by unit, ranging from 70% of students for the unit on blocks to 89% of students for the unit on magnets. Average child pre and post-scores for each of the six ECHOS units are depicted in the graph below. Average post-scores ranged from 87% to 92%.
Click here to download the complete Year 1 Evaluation Report.
Great Lakes Science Center, Cleveland, Ohio
PNC Grow Up Great: Year 2 Final Evaluation Report, August 2014
Executive Summary
The Great Lakes Science Center (GLSC) partnered with Catholic Charities Head Start (CCHS) to support quality science programming for young children and their families. This is the second year that GLSC has trained early childhood educators to implement the Early Childhood Hands On Science (ECHOS) curriculum from Frost Science and the first year of CCHS’ participation.
Programming included professional development (PD) sessions and an individual classroom coaching session for each of the 26 lead teachers and their assistants. Supervisors also received targeted PD sessions. Each classroom received an in-class presentation from GLSC staff and enjoyed three field trips. Families were invited to two family events, received newsletters with suggestions for at-home science activities and a family pass to visit the science center.
The impact of this year’s project was evaluated by collecting data through teacher surveys, family surveys, teacher interviews, classroom visits and feedback from project leads. Among the factors that key GLSC and CCHS staff identified as contributing to the project’s success were the organization-wide buy-in by CCHS staff with an administrator who was very involved with the implementation, the purposeful alignment of the excellent ECHOS curriculum with CCHS’ HighScope curriculum and the engagement of site supervisors.
The results of the project were very impressive. 80% of the teachers interviewed provided examples of their classroom science activities that were rated in the “higher quality” range–twice the rate for comparison teachers and a marked increase over project teachers’ own ratings from the beginning of the year. Classroom environment scores for science approached the “excellent” range.
Teachers reported greater confidence in their skills and knowledge related to early science teaching. They are offering quality science activities with increasing regularity, with all teachers offering science activities at least weekly and 50% reporting daily activities.
Supervisors reported providing more frequent feedback related to science by the end of the year, with half of the supervisors providing feedback on a monthly basis or more. Teachers rated the supports for their science teaching highly and are using them more frequently, with the materials provided by GLSC being used most often.
84% of project families attended a science event with their child. More than half of the project families reported doing a recent at-home science program with the child. Both of these rates were slightly lower than the rate for comparison families. However, of those families who did at-home activities, project families reported doing so more frequently.
Click here to download the complete Great Lakes Evaluation Report.