A guide to the Georgia student growth model

A Guide to the Georgia Student Growth Model

Table of Contents
Introduction................................................................................................................................. 2 Growth vs. Value-Added ............................................................................................................ 2 Student Growth Percentiles......................................................................................................... 2 Understanding Percentiles .......................................................................................................... 4 Combining Growth Percentiles................................................................................................... 5 Interpreting Student Growth Percentiles..................................................................................... 5 Student Growth Levels ............................................................................................................... 5 Calculating Student Growth Percentiles ..................................................................................... 6 Academic Peers........................................................................................................................... 6 Priors ........................................................................................................................................... 7 Assessment Inclusion.................................................................................................................. 8 Retests ......................................................................................................................................... 8 Missing Data ............................................................................................................................... 8 Growth Over Time (Baseline-Referenced Growth Percentiles) ................................................. 8 Growth to Proficiency (Growth Projections and Growth Targets) ........................................... 10 Accessing Data.......................................................................................................................... 11
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 1 of 11 All Rights Reserved

A Guide to the Georgia Student Growth Model
Introduction
The Georgia Department of Education (GaDOE) is implementing the Georgia Student Growth Model (GSGM) in order to provide an additional perspective of student learning improve teaching and learning and inform accountability and educator effectiveness. Historically Georgia s assessment system has only enabled educators and other stakeholders to ask questions such as "What percentage of students met the state standard " Or "Did more students meet the state standard this year compared to last year " The GSGM will allow Georgia to move beyond questions about status to ask critical growth-related questions such as
Did this student grow more or less than academically-similar students Are students growing as much in math as in reading Are students on track to reach or exceed proficiency
The GSGM provides a wealth of rich information on student school district and state performance on Criterion-Referenced Competency Tests (CRCTs) End of Course Tests (EOCTs) and beginning in 2014-2015 the Georgia Milestones Assessment System. In addition to providing student-level diagnostic information and improving teaching and learning the GSGM will work in conjunction with other factors as part of the state s accountability system the College and Career Readiness Performance Index (CCRPI) and serve as one of multiple indicators of educator effectiveness with the Teacher Keys Effectiveness System (TKES) and the Leader Keys Effectiveness System (LKES).
Growth vs. Value-Added
A growth model describes the change in student achievement across time. A growth model becomes value-added when the growth is attributed to an entity (a teacher a school etc.). In many models the value-added is the difference between predicted student performance and actual student performance. These models use information about a student (prior achievement demographic information etc.) to predict how that student will perform. The student s actual performance is then compared to his or her predicted performance. The difference is considered value-added. The GSGM does not predict performance rather it describes observed student growth.
Student Growth Percentiles
Georgia is implementing the Student Growth Percentile (SGP) methodology as its growth model. SGPs describe a student s growth relative to other students statewide with similar prior achievement (students who have the same score history). An SGP not only shows how an individual student is progressing from year to year but it can also describe how groups of students schools districts and the state are progressing. Growth percentiles range from 1 to 99 with lower percentiles indicating lower academic growth and higher percentiles indicating higher academic growth.
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 2 of 11 All Rights Reserved

SGPs do not require a vertical assessment scale in order to describe student growth. A vertical or developmental scale is a continuous scale spanning multiple grade levels in the same content area. The GSGM is not specifying how many scale score points a student improved from year to year. Rather the GSGM describes growth in terms of how a student performed this year relative to other students who have a similar academic history. A common concern with growth and value-added models is the potential for what has been termed "floor or ceiling effects." Floor and ceiling effects refer to the inability to adequately define or distinguish really low and really high student growth. Analyses reveal that Georgia does not have such effects with the GSGM. Figure 1 demonstrates this lack of floor and ceiling effects using actual GSGM results. Figure 1 Growth and Prior Achievement
Figure 1 plots current growth for 5th-grade mathematics students (2012) on the x-axis against these students 4th-grade mathematics scale scores from the prior year on the y-axis (2011). As the figure demonstrates all students regardless of their achievement level are able to demonstrate all levels of growth.
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 3 of 11 All Rights Reserved

Growth percentiles represent how a student performed this year relative to academically-similar students. While there are a few students statewide who continuously score at the top of the assessment scale range there is enough variability in scale scores to produce growth percentiles (i.e. the top row of bubbles in Figure 1). Additionally even students who score at the top of the assessment scale range year after year must "grow" in order to do so. Therefore high-performing students have the ability to demonstrate all levels of growth just as students who are struggling. It is important to remember that demonstrating low growth does not mean a student is low achieving. Even very high-achieving students will demonstrate low growth if they scored lower on the current assessment when compared with other high-achieving students. Therefore it is always important to consider both status achievement and growth.
Understanding Percentiles
One of the most common applications of percentiles is pediatric height and weight charts. When children go to the pediatrician for their well visits the pediatrician measures their height and weight and describes the measurements in terms of percentiles. For example a child may be at the 75th percentile for height. That means that compared to other children his age he is taller than 75% of them. The same concept is used in the GSGM to refer to academic growth. A student at the 75th percentile grew more than 75% of his or her academic peers.
Figure 2 is a visual representation of a normal distribution commonly referred to as a bell curve. Most naturally-occurring variables (such as height or weight) have this distribution where more observations are clustered around a mean with fewer observations far from the mean. This visualization is useful for describing percentiles.
Figure 2 Normal Distribution
50th percentile
50% 50%
A distribution for example of height weight or academic achievement.
For any given percentile a proportion of the distribution falls below it and the remaining proportion of the distribution is above it. Using the 50th percentile as an example 50% of the distribution falls below the 50th percentile and 50% of the distribution is above the 50th percentile. Expressing student growth in terms of percentiles is very informative it provides perspective on what is typical growth.
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 4 of 11 All Rights Reserved

Combining Growth Percentiles
While SGPs are produced for individual students there are multiple ways of combining SGPs to summarize the growth of a group of students (such as for a classroom school or system). One method of combining SGPs for a group of students is to utilize a median. A median is the numerical value separating the higher half of the data from the lower half. In other words it is the middle value in an ordered list. A second method of combining SGPs for a group of students is to utilize a mean. A mean is the sum of the values divided by the number of values. It is often referred to as an average. A third method of combining SGPs for a group of students is to utilize the percent of students demonstrating typical or high growth.
Interpreting Student Growth Percentiles
A fictional student named Anna will be used as an example for interpreting student growth percentiles. Anna has a 6th-grade reading growth percentile of 54. This means that Anna grew at a rate greater than 54% of academically-similar 6th-grade students in reading (Figure 3). The median 6th-grade reading growth percentile for Anna s school is 65. This means that the typical 6th-grade student in Anna s school grew at a rate greater than 65% of academically-similar students in reading. Additionally Anna grew at a lower rate in reading compared to other 6thgrade students in her school on "average."
Figure 3 Interpreting Student Growth Percentiles

Lower growth

Typical growth

Higher growth

54 Anna

65 Typical Student

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percentage of Students

Student Growth Levels
Information about the relationship between student growth and status-based achievement were used to set the following student growth levels
Low 1-34 Typical 35-65 High 66-99 Analyses show that a student who begins Grade 3 scoring just at "Meets" and demonstrates consistent 35th percentile growth across grades likely will end Grade 8 also scoring just at "Meets." A student who begins Grade 3 scoring just at "Meets" and demonstrates consistent 65th
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 5 of 11 All Rights Reserved

percentile growth across grades likely will end Grade 8 having made significant progress towards scoring "Exceeds." Thus 35 and 65 were used as the cut points for the three student growth levels which could be interpreted as
A student who demonstrates low growth generally will struggle to maintain his or her current level of achievement.
A student who demonstrates typical growth generally will maintain or improve academically.
A student who demonstrates high growth generally will make greater improvements academically.
Calculating Student Growth Percentiles
SGPs are statistical regression-based quantities used to characterize the growth of students on state-mandated assessments. SGP calculations utilize quantile regression with b-spline smoothing to create growth norms that model the relationship between students current and prior achievement scores. Specifically for each cohort of students taking the assessment in the same content area and grade quantile regression is used to create 100 conditional percentiles for each student based upon their own scale scores. B-spline smoothing is used in conjunction with the quantile regression analyses to model any non-linearity in the distribution of student scale scores particularly at the high and low end of the assessment scale. The coefficient matrices derived from the analyses relate prior and current achievement across for students across the entire achievement spectrum. These matrices can be calculated each year so that growth norms are annually updated or can be fixed to a baseline period and used with annual data to allow for growth comparisons across years fixed to the same growth norms.
Please refer to Betebenner D.W. (2011). A Technical Overview of the Student Growth Percentile Methodology Student Growth Percentiles and Percentile Growth Projections/Trajectories. Dover New Hampshire The National Center for the Improvement of Educational Assessment. available at http //www.gadoe.org/Curriculum-Instruction-andAssessment/Assessment/Documents/SGP_Technical_Overview.pdf for more detailed information on the technical calculation of student growth percentiles.
Academic Peers
Key to understanding the GSGM and SGPs is understanding the term academic peers. Academic peers are students enrolled in the same grade and content area or course statewide with similar prior academic achievement (academic history). In other words they are students that had the same scores on prior state assessments.
Figure 4 is a visual representation of academic peers. While this figure is simplified to demonstrate only one prior score it is useful to describe the concept of academic peers. One axis (left) plots 2011 scale scores (prior score) while the other axis (right) plots 2012 scale scores (current scores). Academic peers are all of the students with a similar prior score (in this case all of the students who scored 760 on last year s assessment). A student s growth percentile describes how he performed on the current assessment relative to his academic peers. In this
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 6 of 11 All Rights Reserved

example the student s 2012 scale score of 800 places him at the 75th percentile. In other words the student grew more than 75% of his academic peers (those students who had the same scale score in 2011). It is important to note that the 75th percentile is actually derived from all academic peer groups. While the example student s current score of 800 placed him at the 75th percentile a student with a prior score of 850 would have needed a current score of approximately 860 to demonstrate 75th percentile growth. Figure 4 Relationship Between Prior and Current Achievement
There are potentially thousands of academic peer groups as many as there are combinations of scores. A commonly asked question is "Can I see a list of students in a particular peer group " A list of peers however is not what is used to calculate a student s growth percentile. The model uses quantile regression to describe the curvi-linear relationship between prior scores and current scores (see Calculating Student Growth Percentiles). That analysis results in a look-up table that relates prior achievement to current achievement. Using this look-up table any combination of prior scores can be plugged in to obtain an achievement distribution that is dependent on those prior scores. Using that distribution and the current score a student s growth percentile can be identified.
Priors Priors are the historical assessment scores being used to model growth. The GSGM uses two years of prior test data however one year is used when two years are not available. For example growth percentiles for an 8th-grade student who just took the 8th-grade CRCT would have his or her 7th- and 6th-grade CRCT scores as priors. An immediate consecutive prior (prior from the previous year) is required to produce a growth percentile. For example 4th-grade CRCT scores are required to produce 5th-grade growth percentiles. Because one prior is required to produce growth percentiles students who do not
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 7 of 11 All Rights Reserved

have a prior including students new to Georgia will not receive a growth percentile. Similarly even though there is a 3rd-grade CRCT 3rd-grade students will not receive growth percentiles as they do not have a prior.
In addition to prior achievement growth percentile calculations for EOCTs also depend on test sequence and timing (i.e. year taken). SGPs will be produced for all sequences for which there are a sufficient number of students to model growth reliably. This includes students who repeat EOCT courses or take them on a block schedule. For uncommon sequences with few students (e.g. students who were in the 8th grade in 2011 and took US History as 9th-graders in 2012) those students will not receive growth percentiles.
Assessment Inclusion
Student growth percentiles are produced for the CRCT (grades 4-8 reading English/language arts math science and social studies) and EOCTs (Physical Science Biology 9th-Grade Literature/Composition 11th-Grade Literature/Composition US History Economics/Business/Free Enterprise Mathematics I Mathematics II GPS Algebra GPS Geometry Coordinate Algebra and Analytic Geometry). As Georgia transitions to Georgia Milestones SGPs will be produced for end of grade assessments in grades 4-8 in English/language arts mathematics science and social studies and end of course assessments.
Retests
Beginning with the 2012-2013 school year retest data is not longer utilized in the growth model. This means that SGPs should be interpreted as representing students first (main) attempt on a state-mandated assessment for a grade and content area or for an EOCT course.
Missing Data
Some growth/value-added models will impute missing data meaning they generate a plausible estimate of what a missing test score would be based on the test scores of similar students. The Georgia Student Growth Model does not impute or estimate missing data. If a student does not have at least one immediate consecutive prior (prior from the previous year) a growth percentile will not be produced.
Growth Over Time (Baseline-Referenced Growth Percentiles)
Cohort-referenced student growth percentiles describe a student s growth relative to academically-similar students in the state in a given year. With these SGPs student and school growth is relative to the state. Cohort-referenced SGPs can continue to be reported during an assessment transition.
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 8 of 11 All Rights Reserved

SGPs can also be anchored to a baseline enabling the comparison of statewide growth from ear to year. With baseline-referenced SGPs a student s growth is relative to academically-similar students from the baseline years. All students can demonstrate lower or higher growth than students in the baseline. These SGPs cannot be reported during an assessment transition. New baselines can be set once there are several years of Georgia Milestones implementation.
Figure 5 is a visualization of the SGP distribution without a baseline (cohort-referenced). With cohort-referenced SGPs the median SGP for the state will be 50 every year half of students would be below 50 and half would be above 50.
Figure 5 Cohort-Referenced SGP Distribution (No Baseline)
State Median This year 50 Next year 50 Two years 50 etc.

Without setting a baseline the state median will always be 50 with half of students below 50 and half above 50.

Establishing a baseline allows the state to observe change in overall student growth over time. Figure 6 is a visualization of the SGP distribution with a baseline.

Figure 6 Baseline-Referenced SGP Distribution

State Median This year 50 Next year 55 Two years 60

With setting a baseline the state median can change from year to year representing statewide change in growth over time.
It is useful to consider this concept in the context of pediatric height and weight charts. When a 7-year-old boy s height is measured and his height percentile determined the doctor does not wait for all 7-year-old boys to be measured that year to determine the percentile. Rather information about 7-year-old boys from prior years is used to establish that relationship. The same concept is applied in the GSGM. The relationship between prior achievement current achievement and growth from prior years is used to establish the relationship.
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 9 of 11 All Rights Reserved

The baseline is an average of at least four years of data in order to allow for a more stable comparison. As of 2013 all CRCT content areas are baseline-referenced and all EOCTs except mathematics (Math I Math II GPS Geometry and Coordinate Algebra) are baseline-referenced.
Growth to Proficiency (Growth Projections and Growth Targets) In addition to describing observed growth the GSGM will also provide information on possible future growth in the form of growth projections and growth targets. SGPs analyze historical student assessment data to model how students performed on earlier assessments how they performed on later assessments and what level of growth they demonstrated in between. This information is used to create growth projections and growth targets for each student. The growth targets tells us based on where students are now how much they need to grow to Meet or Exceed expectations on the next assessment. The growth projections tell us for all levels of growth where a student may score on next year s assessment. Growth projections and targets will not be provided during the first year(s) of Georgia Milestones implementation. Several years of implementation will be required to provide projections and targets. Figure 7 is a graphic demonstration of growth projections and targets. The growth projection concept is illustrated by the multi-colored fan. This year the example student scored just above Meets. The fan indicates where on the assessment scale the student may score next year depending on the level of growth he demonstrates. For example if he demonstrates very low growth 1st percentile he will likely score Does Not Meet. If he demonstrates very high growth 99th percentile he will likely score Exceeds. The growth targets expressed as SGP values tell us specifically what level of growth he must demonstrate to score Meets or Exceeds. In other words the growth projection tells us "if you grow this much you will score here" while the growth target tells us "to score here you must grow this much." Figure 7 Growth Projections and Growth Targets
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 10 of 11 All Rights Reserved

Accessing Data Students and parents receive student growth reports that provide information on their achievement and growth. Districts schools and teachers can access their growth model data using the Statewide Longitudinal Data System (SLDS). The general public can access schooland district-level results through the public growth data tool available on the GSGM website (gsgm.gadoe.org).
Georgia Department of Education Dr. John D. Barge State School Superintendent
July 2014 Page 11 of 11 All Rights Reserved