Georgia's testing program, 2007

What Georgia educators need to know about...

Georgia's Testing Program KathyCox,StateSuperintendentofSchools
"We will lead the nation in improving student achievement." Kathy Cox, State Superintendent of Schools

A Message from the Superintendent:
No one understands the importance of testing more than our teachers. A good testing program, combined with a strong curriculum, can pave the way for targeted, focused instruction that will help Georgia improve student achievement like never before in our history.
I hope this newsletter will provide the answers to some of the questions you have about our testing program and some of the questions you hear from parents and the public.
On these pages, you'll find in-depth details about each of the state's standardized tests, including how the exams came into being, how the questions are developed, and the technical quality of the tests.
Please take some time to read through this newsletter and keep it nearby should you have questions in the future. And, as always, please feel free to contact our Testing Division at (404) 656-2668 should you have any further questions.
I am so grateful for the work that our teachers and administrators do every day for the children of Georgia. It is an honor to be your colleague.
State Superintendent of Schools

Overview of Test Development
The first step in test development is to determine the purpose of the test. In some cases, the state legislature determines the purpose; in other cases, the GaDOE holds focus groups around the state to solicit input from stakeholders. Once the purpose of the test is established, committees of Georgia educators are formed to review the curriculum and establish how the concepts, knowledge, and skills will be assessed. The results of this are a test blueprint and test specifications in which the committees indicate which standards can and will be measured and how they will be represented on the assessment.
Additional specifications are typically produced before item writing can begin. Through careful review of the state curriculum, content domain specifications are produced. These indicate how specific standards or elements of the curriculum will be grouped into domains or strands. Moreover, test item specifications are produced, which give additional detail about what kinds of items will be written. This document typically identifies the item format, content scope and limits, and cognitive complexity. For example, item specifications for a reading test may address the genre, complexity, and/or length of literary passages to be produced. All of these activities are conducted by the GaDOE and the assessment contractor with substantial involvement by curricular specialists and Georgia educators.
Once this is accomplished, items are written by qualified, professional assessment specialists specifically for Georgia tests. Committees of Georgia educators review

Georgia'sTesting Program

the items for alignment with the curriculum, suitability, and potential bias or sensitivity issues. The review committees have the authority to accept the item as is, revise it, or reject the item.
Items that are accepted are placed on field tests. Field tests, which are trial runs of the test items, are designed to help ensure that the items function appropriately and are not confusing for students. Typically, this is accomplished by embedding field test items in the operational test. This is a commonly used and well regarded practice that ensures the field test items are taken by a representative group of motivated students under standard conditions.
After the items have been field tested, another committee of Georgia educators is formed to review the items again along with the data from the field test. The committee reviews how the item performed in terms of how many students selected the correct answer and how many students selected each incorrect answer. The review also includes an analysis of how different groups of students performed to detect potential bias (i.e., did the item appear to favor one group of students over another?). Once again, the review committees have the authority to accept items as is, revise items for re-field testing, or reject items. Accepted items are then banked for future inclusion on an operational test form. Only after items have been field tested and approved by Georgia educators do they appear on a test form.
The next stage of test development consists of developing the actual test form that students will take. Items are carefully selected for a test form based on the blueprint developed by Georgia educators. Putting together a test form requires consideration of both content and statistical data. Each form of a test must assess the same range of content as well as carry the same statistical attributes.
When multiple test forms are used in the same administration or when a test is given in subsequent administrations (e.g., year to year tests), they must be equated. Equating refers to a statistical procedure to make sure that the tests are of equal difficulty. This is critical because it ensures that students are always held to the same standard. Additionally, it permits the interpretation of differences in test performance as the

result of changes in student achievement as opposed to fluctuations in the properties of the test form.
When a test is administered for the first time, standards must be established for the test. The standard setting process is the means by which educators decide what number of items a student must get correct in order to meet or exceed expectations.
The final stage in test development is to produce scores and distribute results. Scores are typically reported as scale scores and performance levels. A scale score is based on the raw score (i.e., number of items correct) on a test. Changing raw scores to scale scores is analogous to converting from the centigrade scale to the Fahrenheit scale to report temperature. Scale scores are commonly used in large assessment programs. As an example, scores on the SAT, the widely used college entrance exam, are reported on a scale ranging from 200 800. Each time a new version of the SAT is administered, the raw scores are converted to this same scale in order to take into account any differences between various forms of the tests. This means that results can be consistently and meaningfully interpreted by students, parents, and educators.
Georgia's New Math
The Georgia performance standards for mathematics have been designed to achieve a balance among concepts, skills, and problem solving. The curriculum stresses rigorous concept development, presents realistic and relevant tasks, and keeps a strong emphasis on computational skills. At all grades, the curriculum encourages students to reason mathematically, to evaluate mathematical arguments both formally and informally, to use the language of mathematics to communicate ideas and information precisely, and to make connections among mathematical topics and to other disciplines. The implementation of this curriculum will require that mathematics classrooms at every grade be student-focused rather than teacher-focused. Working individually or collaboratively, students should be actively engaged in inquiry and discovery related to the world around them.
Assessments aligned to the GPS will reflect the goals of the curriculum. Problem-solving, reasoning, and communication of mathematics through multiple representations will be incorporated in items that require higher levels of critical thinking and mathematical literacy than have been expected of students in the past.

2

Georgia'sTesting Program

Georgia's Criterion-Referenced Competency Tests (CRCT)

T he CRCT program is designed to measure student acquisition of the knowledge, concepts, and skills set forth in the state curriculum. The testing program serves a dual purpose: 1) diagnosis of individual student and program strengths and weaknesses as related to instruction of the Georgia Performance Standards (GPS) and the Quality Core Curriculum (QCC) where applicable and 2) a measure of the quality of education in the state. The CRCT are administered in reading, English language arts, and mathematics in grades 1-8 with students in grades 3-8 also participating in the science and social studies tests. Assessments and reports yield information on academic achievement at the student, class, building, system, and state level.
Content of the CRCT
The new Georgia Performance Standards provide a deeper and richer curriculum for Georgia students and are being phased-in to replace the QCC. Because CRCT are a curriculum-based assessment program, the tests are being redeveloped to reflect GPS in accordance with the phase-in plan for the new curriculum. The transition plan for the new GPS includes one year of training for school districts and a second year for full implementation of the GPS in the classroom. It is during this second year that the CRCT will transition to GPS and students will be assessed on the new curriculum.
The Testing Division of the Georgia Department of Education begins the redevelopment of GPS-based assessments long before students take the new tests. Typically, it takes two years to generate operational items used to produce student scores. In the 2005-2006 academic year, the GPS-based subjects and grades that were in the implementation and, therefore, assessment phase included English language arts in grades 1-8, reading in grades 1-8, mathematics in grade 6, and Science in grades 6 and 7. The transition to GPS-based assessment for other grade levels and content areas is continuing based on the rollout of the GPS. In the 2006-2007 school year, students will also be assessed on the GPS in mathematics grades 1, 2, and 7 and science in grades 3-5. More details about the CRCT and all other tests based on the state's curriculum are provided in the "Overview of Test Development" article in this newsletter.

To learn more about the content of the CRCT and the transition from QCC to GPS, visit the Department's website at www.gadoe.org. Specifically, the CRCT Content Descriptions provide Georgia educators and the public with specific information about how the curriculum is assessed. The CRCT Content Descriptions are built from the test, item, and domain specifications. They detail the concepts and skills that are assessed on the CRCT, including what standards are assessed under each domain.
Administration of the CRCT
All Georgia public school students enrolled in grades 1-8 are required to participate in the CRCT. The Georgia Department of Education determines a 30-day state testing window in the spring within which school districts have flexibility to select a nine-day testing window. Students that perform at Level 1 (not meeting the standard) in grade 3 reading or grade 5 and 8 reading and math are given an opportunity to attend remediation classes in their districts and then take the CRCT retest in the summer. The state testing window usually spans from the first week in April until the first week in May. Generally, the tests are administered over five consecutive days (three days for first and second grade). Students are tested on one subject per day. Each subject has two test sections that last about 60 minutes each. Teachers in grades 1-2 read each question and answer choices (if applicable) to the students whereas in grades 3-8, students read the questions and answer options independently and work at their own pace.
CRCT Testing Window
Spring 2007: April 2 May 2 Systems select a 9-day window within the state testing dates.
Summer 2007 Retest Dates: May 28 July 27
Performance Levels
Student performance standards for the CRCT are established through a standard setting process in which educators from around the state participate. Educators make recommendations on what scores define categories of student performance. As a result of this process, student scores on the CRCT are reported in the following scale scores and performance levels.

3

Georgia'sTesting Program

Performance Levels: Does Not Meet Standard Meets the Standard Exceeds the Standard

QCC below 300 300-349 350 and higher

GPS below 800 800-849 850 and higher

Reports A variety of reports are made available to assist schools, parents, and districts in analyzing student scores to gauge curriculum strengths and areas needing improvement. The Testing Division works with its CRCT contractor to ensure the accuracy and timely delivery of these reports. To help in the timely delivery, it is essential that districts return tests materials promptly to the contractor .

Once the Testing Division staff and the CRCT contractor have conducted quality control checks, school districts begin to receive their results. The vast majority of districts receive their reports five days from the time their test materials are received by the test contractor. The first available reports are issued in Portable Document Format (PDF) and include Class Rosters and paper copies of Performance Level 1 (Does Not Meet) reports for the "promotion" gateway tests (grade 3 reading and grades 5 and 8 reading and mathematics). Additionally, an electronic file is made available to

each district through the Department of Education's Web Portal. A shipment of makeup results is sent to each district soon after the main shipment of results.
About two weeks later, districts receive printed copies of Class Rosters and all Individual Student Reports. In the early part of July, districts receive all the summary level reports, including school, system, and state performance summaries. They also receive a report of all student populations. An interpretive guide, which provides guidance for interpreting various reports, is posted on the Testing section of the GaDOE website.
Starting in spring 2006, performance on the reading portion of the CRCT was linked to the Lexile scale, a national reading measure. A separate article on Lexiles is included in this newsletter. For more information about the Lexile scale and resources to help educators and parents select literature for the students, visit www.gadoe.org/lexile.aspx.
Additional information on the CRCT may be accessed from the Georgia Department of Education's website: www.gadoe.org.

Assessing the English Language Proficiency of Georgia's English Language Learners: The ACCESS for ELLs

In spring 2006, Georgia administered the Assessing Comprehension and Communication in English State to State for English Language Learners (ACCESS for ELLs) for the first time. ACCESS for ELLs is an English language proficiency test that is administered annually to all English language learner (ELL) students. In accordance with the federal No Child Left Behind (NCLB) Act of 2001, schools are required to annually assess the English language proficiency of their ELL students using a standards-based instrument. The ACCESS for ELLs was developed by a federally funded consortium of states known as WIDA (Worldclass Instructional Design and Assessment). Currently, WIDA consists of fifteen states, of which Georgia is one, and the District of Columbia. WIDA first created English Language Proficiency (ELP) standards that aligned to the consortium member states' academic content standards. The assessment, ACCESS for ELLs, was then developed from the WIDA ELP standards.

Technical assistance for this important initiative has been provided by the Center for Applied Linguistics (CAL). CAL is a leading research organization in the field of language, linguistics, and cultural learning.
In Georgia, the ACCESS for ELLs serves five main purposes:
to determine the English language proficiency level of students;
to provide districts with information that will help them evaluate the effectiveness of their ESOL/ language assistance programs;
to provide information that enhances instruction and learning in programs for ELL students;
to assess annual English language proficiency gains using a standards-based assessment instrument; and
to provide data for meeting federal and state requirements with respect to student assessment.

4

Georgia'sTesting Program

The ACCESS for ELLs series spans five grade level clusters, which include Kindergarten, Grades 1-2, Grades 3-5, Grades 6-8, and Grades 9-12. Students are assessed in the four domains of speaking, listening, reading, and writing. Student results are reported for each of the four domains, with an overall proficiency level denoting the student's English language acquisition. There are six proficiency levels: Entering (Level 1), Beginning (Level 2), Developing (Level 3), Expanding (Level 4), Bridging (Level 5), and Attained (Level 6).

Consistent and comprehensive test administrator training is an essential component of a valid testing program. In order for test scores to have any degree of validity from which meaningful interpretations can be made, tests must be administered in as standardized a manner as possible. Comprehensive training of test administrators is a major component of creating standardization, particularly for assessments that require test administrators to score student responses as the student responds. The ACCESS for ELLs contains items for which the test administrator records a score for student responses based on the provided rubric. It is therefore essential that all test administrators demonstrate the ability to apply the rubric consistently and reliably. The primary mode of training for test administrators is through an online course called Desire 2 Learn (D2L). D2L is sponsored by the University of Wisconsin-Oshkosh. Each component of the ACCESS for ELLs training has an associated quiz, which test administrators must complete. The results of these quizzes are how GaDOE verifies that test administrators are "D2L certified." This means test administrators have demonstrated an understanding of how to apply the rubric consistently and are trained to administer the assessment. To become "D2L certified," the test administrator must pass each quiz with a score of 80% or more items correct.
Students who have exited language assistance services or who are on a monitored status (identified as ELL-M) do not take ACCESS for ELLs. Given that the ACCESS for ELLs is a measure of English proficiency, test administration accommodations may not be provided unless the student qualifies for special education services and the accommodation is required to the student's disability and not his/her English proficiency.

The ACCESS for ELLs is an important component of Georgia's accountability system under No Child Left Behind. While all students identified as ELL must be assessed for Title III accountability, the assessment also serves a crucial role in determination of Adequate Yearly Progress (AYP). For those ELL students new to a U.S. school who have been granted a one-time deferment, the ACCESS for ELLs serves as the reading/English/language arts participation proxy for the Criterion-Referenced Competency Tests (CRCT) and the Georgia High School Graduation Tests (GHSGT). State Board of Education rule allows ELL students enrolling for the first time in a U.S. school to receive a one-time deferment from content area assessments, with the exception of mathematics, if their proficiency in English indicates that testing is not in the best educational interest of the student. Students must participate in the mathematics content assessments of all state-mandated tests. In addition, the deferment does not apply to any End-of-Course Tests (EOCT).

5

Georgia'sTesting Program
Georgia High School Graduation Tests (GHSGT)

Purpose of Tests
Since 1994, Georgia's High School Graduation Tests have served to ensure that
ALL Georgia students have access to a quality education;
Georgia students who earn a Georgia diploma meet performance standards established for graduation by Georgia educators and other stakeholders;
Students, educators, and parents have information about student strengths and areas for improvement; and
Georgia schools have summative data for school improvement.

Overview In 1991, the Georgia legislature passed a law requiring that curriculum-based assessments be administered in grade 11 for graduation purposes. The assessments were phased in between 1994 and 1997. Since 1997, all students seeking a Georgia high school diploma must pass the Georgia High School Graduation Tests (GHSGT) in four content areas as well as the Georgia High School Writing Assessment (GHSWT). The assessments cover a sample of the knowledge and skills that educators agree constitute a comprehensive high school education.
Test Characteristics The GHSGT are multiple-choice tests grounded in state-mandated curriculum, either the Quality Core Curriculum (QCC) or the new Georgia Performance Standards (GPS). The tests help ensure that students have mastered the content necessary to become productive members of an increasingly mercurial and competitive job market, including being successful in post-secondary education. The curriculum standards assessed were selected by Georgia educators based on judgments of what constitutes an appropriate and adequate education for high school students. The main administration of the GHSGT in English/language arts (ELA), mathematics, science, and social studies occurs in the spring of the eleventh grade. Students have three

hours to complete each test, and test lengths are 60 questions for ELA, 70 for mathematics, 80 for science, and 90 for social studies.
GHSGT and GPS Alignment The GHSGT are being aligned to the Georgia Performance Standards according to the GPS implementation schedule. In spring 2006, the Georgia High School Graduation Tests (GHSGT) in English/ language arts and science were dually aligned to the GPS and the QCC. It is important to remember the GHSGT assess core content and skills students have had an opportunity to learn throughout their high school careers. Because Georgia does not mandate course sequence, some eleventh-grade students in spring 2006 had received previous instruction on the core concepts/ skills assessed based solely on the QCC, while others may have been enrolled in English and science courses that are based on the GPS. The Georgia Department of Education worked with Georgia educators to identify those core concepts/skills that are common to both curricula and to develop a plan that will ensure students are tested on concepts they will have had an opportunity to learn whether instruction was provided through the QCC or the GPS. This transitional blueprint has been in effect for two years (2005 2006 and 2006 2007). An exclusively GPS-aligned GHSGT in English/language arts and science will be implemented in spring 2008,

6

Georgia'sTesting Program

at which time all first-time test takers (i.e., eleventh graders) will have had only GPS instruction during their high school careers. A similar plan will be carried out in social studies, with dually aligned tests administered in spring 2008 and 2009 and an exclusively GPS-aligned test given in spring 2010. A GPS mathematics test will be administered in spring 2011. As tests are exclusively aligned to GPS in each content area, new standard settings will take place and new cut scores and scale score ranges will be established.

Multiple Assessment Opportunities The GHSGT are considered high stakes because a student must pass these tests, and the Georgia High School Writing Test (GHSWT) as one requirement for graduation. Therefore, students that do not pass on the first attempt have multiple opportunities to receive additional instruction, retest, and qualify for graduation before the spring of the twelfth grade. Students are only required to retake the content area(s) they have not passed.

Assessment Opportunities
Grade 11 Fall (September) Grade 11 Spring (March/April) Grade 11/12 (July) Grade 12 Fall (September) Grade 12 Winter (November) Grade 12 Spring (March/April)

Writing
First Retest Retest Retest
Retest

Language Arts, Mathematics, Science, and Social Studies
First Retest Retest Retest Retest

Students that do not pass all the required tests but meet all other graduation requirements may be eligible for a Certificate of Completion (Performance or Attendance) or a Special Education Diploma. Students who have left school with a Certificate of Performance or a Special Education Certificate may return to attempt the graduation test(s) as often as necessary to qualify for a high school diploma. Local school districts decide whether students who have not passed one or more of the tests may march during graduation ceremonies.

The Georgia State Board of Education allows students who meet specific criteria to apply for a waiver or variance from certain GHSGT requirements. Students submit a packet documenting the rationale for a waiver or variance. A committee reviews the request and submits it to the State Board of Education for a decision. Information and guidelines about the Waiver and Variance policy (Board Rule 160-1-3-.09) appear on the Georgia Department of Education webpage at www.gadoe.org.

Performance Levels and Scale Scores for the GHSGT

The performance levels for the GHSGT are Pass Plus, Pass, and Fail. The scale score range is 400-600, with 500

as passing. Scores are reported to systems following each administration and are reported to the public on an an-

nual basis. The Pass and Pass Plus scores for all four content areas appear in the following chart:

Pass

Pass Plus

English Language Arts

500

538

Mathematics

500

535

Science

500

531

Social Studies

500

526

Georgia High School Graduation Tests and No Child Left Behind (NCLB) In the spring of each year, the English/language arts and mathematics tests are enhanced to meet the standards of NCLB. The enhanced tests are used to establish Adequate Yearly Progress (AYP) at the school, system, and state level. Students who take the enhanced test do not have different performance expectations for diploma eligibility. In other words, the standards used to determine the PASS and PASS PLUS levels have not changed nor has the degree of difficulty in meeting these two standards changed as a result of the enhanced test version. A scale score of 500 is still required to pass each of the GHSGT content area tests.
7

Georgia'sTesting Program

Scale Scores for Performance Levels

Student Accountability (Diploma)
School, System, & State Accountability (AYP)

Fail Pass Pass Plus Basic Proficient Advanced

GHSGT - ELA
Below 500 500-537 538 and above Below 511 511 537 538 and above

GHSGT -Mathematics
Below 500 500-534 535 and above Below 516 516- 524 525 and above

Proficiency scores, or cut scores, for the enhanced ELA and mathematics were set by committees of Georgia educators in April 2004 and approved by the State Board of Education in May 2004. These scores were used to set Annual Measurable Objective (AMO) targets for determining AYP beginning in 2004. Schools, systems, and the state are accountable for the percent of students meeting the Proficient target each year. The target will increase each year according to AMO targets.

Anticipated Improvements in the GHSGT Several improvements are being added to the GHSGT reporting system. In spring 2006, electronic Pass-Fail rosters were provided to systems in an effort to report student results earlier. This practice will continue with the addition of electronic student individual reports in spring 2007. Beginning in spring 2007, student score reports and school and system summary data will be available early May, and electronic student data files will be provided to each school system. Also, the winter 2007 retest date will be changed to January to allow for additional instruction between the fall and winter testing.

8

Georgia'sTesting Program

End-of-Course Tests (EOCT)
In 2000, the state legislature mandated the development and adoption of end-of-course assessments in high school content. The program has two established purposes to ensure all Georgia students have access to a rigorous curriculum that meets high expectations and to provide information to improve student achievement through effective instruction of the standards in the state-adopted curriculum.
Content The End-of-Course Tests are administered in eight subject areas:
Ninth Grade Literature American Literature Physical Science Biology Algebra Geometry U.S. History Economics/Business/Free Enterprise
As a course transitions to the new Georgia Performance Standards (GPS), the EOCT for that course is revised. During the 2004-05 school year, the new GPS high school curriculum in English/language arts and science began to roll out, and the GaDOE began redeveloping and realigning the EOCT in these areas. These GPS-based Language Arts and Science EOCT were administered for the first time in winter 2005. As the revised EOCT become operational, new standards are set, and the range of scale scores will change.
The GaDOE is in the process of redeveloping the Social Studies EOCT to be aligned with the new U.S. History and Economics curricula, which will be implemented in the 2007-2008 school year. In July of 2007, the Mathematics EOCT will begin redevelopment in preparation for the transition to the GPS in the 20082009 school year. The new Mathematics EOCT will be administered upon completion of Math I/Accelerated Math I and Math II/Accelerated Math II courses in the GPS.

In the Testing section of the GaDOE website, students and teachers can find descriptions of the content for each test, previously used "released" tests, as well as study guides useful in preparation for all of the EOCT.
Students To Be Tested State Board of Education Rule mandates that any student, regardless of grade level, enrolled in a course for which there is an EOCT should participate in the assessment. The score contributes 15% to the student's final course grade with the remaining 85% comprised of classroom performance. GaDOE receives many requests for clarification on this policy regarding which students should be tested. The following are specific examples of students who must take the EOCT:
If a student is taking a course to complete the state requirement for Algebra I, then that student must take the Algebra EOCT, regardless of the course title assigned by the school system.
Students enrolled in a school district who are dually enrolled in a college course, an online course, or a correspondence class and are receiving credit towards graduation for an EOCT course must take the appropriate EOCT
Middle school students enrolled in an EOCT course, whether or not they receive high school credit for the class, must take the EOCT.
Students who are repeating a course or are in credit recovery (e.g., Novanet) even if they passed the EOCT but failed the course previouslymust retake the EOCT.
To earn credit for a course requiring an EOCT, a student enrolling from a non-accredited or home-school program must take and pass the corresponding EOCT. Local board policy will determine grade assignment.
Administration EOCT, each administered in ninety minutes, are given over one or two school days, with dates for each test selected by the local school system. The EOCT is administered three times during the year: winter, spring, and summer. In addition, mid-monthly administrations are also offered. There were approximately 880,000 EOCT administered to students throughout the state

9

Georgia'sTesting Program

during the 2005/2006 school year, and about 80,000 of these were administered online. Schools have the option of administering the test either online or with paper-and-pencil tests.

Performance Levels Cut scores for performance levels vary according to whether the test is based on the Quality Core Curriculum (QCC) or GPS.

EOCT Dates for 2006-2007

Winter 2006: Spring 2007: Summer 2007:

November. 27 January 12 April 23 June 1 June 18 July 27

Online Mid-Month Assessments: August, September, October, November, February, and March

Reports and Use of Results Test results are reported within five days from the receipt of the answer documents. Results from online administrations are usually received within three days.

For the four QCC-based assessments; Algebra I, Geometry, U.S. History, and Economics; the performance levels are defined as:
Does Not Meet = Below 600 Meets = 600 to 629 Exceeds = 630 and above
For the four GPS-based assessments; 9th Literature, American Literature, Biology, and Physical Science; the performance levels are defined as:
Does Not Meet = Below 400 Meets = 400 to 449 Exceeds = 450 and above

Electronic Class Rosters and, if requested by the local system, Individual Student Reports are posted to a secure website for System Test Coordinators. System Test Coordinators can arrange for each school's principal to view and download their school's data.

Summary reports are sent to each system after all testing is completed and the results have been tabulated for the state. In addition to state performance data, these reports include System Summary Reports, a School Summary Report, and a Class Summary Report. Content Area Summary Reports are generated at the system, school, and class levels. For more details on these reports and how to interpret them, look for the Interpretive Guide on the EOCT section of GaDOE's website.

Student Scores Student scores are reported in scale scores, by performance levels, and with grade conversions. This grade conversion is to be used in calculating the student's final grade because State Board Rule mandates that the EOCT be counted as 15% of the student's final course grade. If a student is enrolled in a year-long course, the EOCT should be averaged in as 15% of the second semester course grade.

10

Georgia'sTesting Program

Assessing Kindergartners in Georgia Public Schools

Current Test: Georgia Kindergarten Assessment Program-Revised (GKAP-R)
Georgia Law requires that all children enrolled in Georgia public school kindergarten programs be assessed for first-grade readiness. To comply with state statute, the State Board of Education adopted the 1990 Georgia Kindergarten Assessment Program (GKAP) as the designated kindergarten assessment for all Georgia public schools. The Georgia Department of Education (GaDOE) significantly modified and improved the original assessment. This was implemented in fall 1998 as the Georgia Kindergarten Assessment ProgramRevised (GKAP-R).

The primary purpose of GKAP-R is to provide cumulative evidence of a student's readiness for first grade. Thirty-two Georgia kindergarten standards are measured using performance-based assessment activities.

Window 2: January February
Reassessment of remaining Window 1 activities not accomplished

Students are assessed in the domain areas of literacy, mathematics, and social/emotional development in a variety of one-on-one, small group, and large group instructional settings throughout the kindergarten year.

Assessment of 14 new activities, if not already presented and accomplished
Begin Window 3 activities as appropriate for individual students

State Board of Education Rule specifies that only certified teachers of kindergarten or first-grade students that have been trained in the use of the GKAP-R shall administer the assessment. GKAP-R is administered during three Windows of Assessment.
Recommended Windows of Assessment guidelines are outlined in the GKAP-R Administration Manual and are as follows:
Window 1: First two weeks of the school year
Ten baseline activities are administered
All kindergarten students are assessed unless specified in an IEP or IAP
Baseline assessment data is recorded and submitted for scoring

Window 3: March April
Reassessment of remaining Window 1 and Window 2 activities not accomplished
Assessment of eight new activities, if not already presented and accomplished
Completion of third administration of all activities as needed by individual students
Individual ratings of all activities are recorded and submitted for scoring beginning April 15 and no earlier than five weeks prior to the conclusion of a school year, whichever comes first as determined by a school system's calendar.
Performance assessment rubrics specifically define student progress and attainment for each activity and appear on every GKAP-R activity sheet. Student

11

Georgia'sTesting Program

achievement is recorded on the student progress profile in one of three rating areas for each administration of every content standard as follows:
1. Not Evident (NE) little to no evidenced skill as defined by the rubric
2. In Progress (IP) developing skill with evidence of application as defined by the rubric
3. Accomplished (AC) proficient skill development and application as defined by the rubric
GKAP-R results; in concert with teacher recommendation, parent/guardian input, and other relevant pieces of information; are used in determining a student's readiness for first grade and in making individual student placement decisions. Final GKAP-R individual, classroom, school, district, and state reports are distributed annually to school systems by the Georgia Department of Education.

The Future: Georgia Kindergarten Inventory of Developing Skills (GKIDS)
Redevelopment of the Georgia KindergartenAssessment began January 2007. The new assessment will be known as GKIDS (Georgia Kindergarten Inventory of Developing Skills). With guidance from broad-based groups of educators and experts from around the state, GKIDS will fully align to the GPS and represent the most accurate and appropriate assessment of first-grade readiness possible. The new assessment will be field tested to ensure it is valid and reliable. Moreover, new standards will be established and reports will be developed to provide useful information to parents and educators.
GKIDS will provide stakeholders information to determine how well kindergarten students are acquiring the knowledge and skills described in the GPS. This will enable educators to provide early remediation to kindergarten students who require support to be successful in the first grade and beyond. Moreover, it will enable administrators and policy makers to evaluate the effectiveness of statewide kindergarten programs.

12

Georgia'sTesting Program

Lexiles

As part of the Georgia Department of Education's mission to develop good readers, Lexile measures were incorporated into our Criterion-Referenced Competency Tests (reading test) and Georgia High School Graduation Tests (English/language arts test) programs. More than a test score, a Lexile measure is a tool to assist students, their parents, and teachers in selecting material that matches their students' current reading comprehension.

What is a Lexile Score? Lexile is a standard score that matches a student's reading ability with difficulty of text material. A Lexile can be interpreted as the level of book that a student can read with 75% comprehension. Experts have identified 75% comprehension level as offering the reader a certain amount of comfort and yet still offering a challenge. Lexiles typically range between approximately 200L and 1700L. Lexile text below 200L represents beginning reading material, and a student's Lexile score may have a number in the 100s or the code of BR (Beginning Reader).

This table shows an example of some book titles that reflect Lexile measures of BR.

ISBN 0152020632 0813620082 0478126123 1558586466 080506205X 0439330173 0399233881 0516020072 0763515337 0673803813

Title "Fire, Fire!" Said Mrs. McGuire "POP" Pops the Popcorn "Who Took the Cake?" 1. 2. 3... What Do You See? 1, 2, 3, Go! 100th Day, The A is for Salad Addition Annie After the Flood Aqua Aqua Aqua

Author Martin Jr., Bill Egan, Bob Medina, Eduardo Bohdal, Susi Lee, Huy Voun Maccarone, Grace Lester, Mike Gisler, David Giles, Jenny Mora, Pat

Lexile BR BR BR BR BR BR BR BR BR BR

How is Georgia able to include a Lexile Score as part of regular assessments? During the 2005-2006 school year, GaDOE and MetaMetrics conducted a study to link existing state tests to the Lexile scale. A representative sample of students from selected districts across the state took a Lexile Linking Test (LLT) a few weeks prior to the regular administration of the GHSGT in English/language arts or the CRCT in reading. By comparing scores on the LLT with either the GHSGT or CRCT, a set of Lexiles to GHSGT or CRCT look-up tables was produced. These look-up tables allow students to get a Lexile score along with their CRCT or GHSGT score without having to take an additional assessment.

Where are Lexile Scores Found? Beginning with spring 2006 testing, if a student takes either the reading CRCT or the English/Language Arts GHSGT, then he or she will receive a Lexile score on the individual student report produced from that administration. Reports for the 2006-2007 school year will be redesigned to make the Lexile information more prominent.

How will knowing a child's Lexile Score help? Lexile scores provide teachers with a valuable tool. Teachers can use a student's Lexile score to identify reading material that best matches a student's reading ability. By finding books that match a student's Lexile score, the teacher can locate material that the student will comprehend while presenting the student with enough challenge to promote growth of his or her reading ability. For more information on using Lexiles in the classroom, please visit these websites:
www.gadoe.org/lexile.aspx The Testing Division of the Georgia Department of Education has created a page dedicated to Lexiles with frequently asked questions. In addition, this site contains a short presentation that focuses on using Lexiles in the classroom and in the home. This presentation is available in both a PowerPoint and PDF format.
www.lexile.com This site is presented by MetaMetrics, the developer of the Lexile Framework, and provides a wealth of information for teachers and families on Lexiles. At this site, you can use the Lexile Book Database to search for books by Lexile ranges.
13

Georgia'sTesting Program

How do you find books for students using Lexiles? First, calculate a student's Lexile range. Add 50 to the student's reported Lexile measure to find the top of the range and subtract 100 to find the bottom; in other words, locate 50L above and 100L below their reported Lexile measure. This range represents the boundaries between the easiest kind of reading material for the student and the hardest level at which he/she can read successfully.
Example: Susie's Individual Student Report shows she has a Lexile measure of 450. Her range would be 350L to 500L. To find reading material that she can read with at least a 75% comprehension level, select books, magazines, or other reading material within this range.
Select reading material within that Lexile range. You may also consider a student's interests and favorite authors. Libraries now have many books that have been tagged with a Lexile score. Ask your school media specialist or public librarian for more information. Additionally, MetaMetrics has a Lexile Book Database that contains tens of thousands of titles. You can search by book title, author, keyword, or Lexile range. Visit the Lexile Book Database at www.lexile.com.

Is there a relationship between Lexiles and grade levels? Often the Georgia Department of Education is asked whether a student is reading on grade level. In the typical classroom, students will exhibit a wide range of reading ability, and the reading materials may reflect a wide range of text difficulty. The intent of the Lexile measure is to match the reader with the appropriately difficult text regardless of the student's grade level.
MetaMetrics has studied typical reader ability at various grade levels as well as typical text demands for each grade. They have compiled a chart of Lexile ranges for readers and for texts at each grade level. The table below shows these ranges along with ranges of Lexiles for the CRCT cut scores for Meets and Exceeds and the cut scores for the GHSGT Pass and Pass-Plus.

Relationship of Lexiles to Grade Level and State Test Cut Scores

Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade HS

Interquartile Range of Lexile Scores for
Students
BR to 300 140 to 500 330 to 700 445 to 810 595 to 910 665 to 1000 735 to 1065 805 to 1100 940 to 1210

Interquartile Range of Lexile Scores for
Textbooks
200 to 400 300 to 500 500 to 700 650 to 850 750 to 950 850 to 1050 950 to 1075 1000 to 1100 1100 to 1300

Range of Lexiles from Meets (Pass) to
Exceeds (Pass-Plus)
BR to 205 130 to 500 410 to 790 570 to 915 650 to 1040 685 to 1120 800 to 1210 805 to 1265 820 to 1105

BR = Beginning Reader

One can see that the range of reader ability is greater than the span of text difficulty. Because the readers' Lexile scores are less than the demand of the textbooks typically found at a grade level, students with Lexiles at the low end of the Reader range, as well as those below the range, will probably experience some difficulty comprehending the text materials typical of that grade level. In most cases, the "meets" cut scores on the CRCT and the "pass" score on the GHSGT fall within the range of text difficulty. The "exceeds" cut scores on the CRCT are typically above the upper limit of text difficulty and the upper bound of typical reader ability.

14

Georgia'sTesting Program Georgia's Online Assessment System

The Georgia Online Assessment System (OAS) is again available for the 2006-2007 academic year. The OAS is a powerful tool that allows teachers to create customized formative assessments to monitor student progress and adapt work toward individual students' needs. Moreover, the system has been enhanced with numerous upgrades intended to make this year the most successful since its inception.

What's New with the OAS: Upgrades

Upgrades are helping to make this tool even more user-friendly with increased utility for all users. Educators, parents, and students will be surprised with the new look and feel; the OAS is both more attractive and intuitive. Previously, the site was divided into regionseach having a different web address. This year, all users will log into one location at www.georgiaoas.org.

Additional Grade Levels and Items The OAS was originally developed for the CRCT. It has been expanded to include items for high school to facilitate student preparation for the Georgia High School Graduation Test and the End-of-Course Tests. All items housed in the OAS are retired items from our state-mandated testing programs. Using retired items guarantees that each item included in the OAS has undergone the same rigorous scrutiny that our test development process employs. With the implementation of our new curriculum, the Georgia Performance Standards (GPS), items have been realigned for the subjects/grades that have transitioned to GPS. This realignment means the items for those subjects and grades now on GPS will display and report not only at the domain but also at the standard and element level.

GPS-aligned subjects/grades include:

Mathematics

Grades 1, 2, 6, 7

English/Language Arts

Grades 1-12

Reading

Grades 1-8

Science

Grades 3-7, 9-12

**Some grades/subjects remain aligned to the previous curriculum, the Quality Core Curriculum (QCC).

Benchmarks The Georgia Department of Education has created benchmark tests for the GPS content areas. Items comprising the benchmarks were selected to be broadly representative of the curriculum, ensuring that the benchmarks mirror the content distribution of the operational tests administered in the spring. These items were realigned from QCCs to the Georgia Performance Standards. The first tests were available on August 15, 2006 and built from the teacher level of the OAS (Level 2). Subsequent benchmarks will be available approximately every 9 weeks and will be created in the system administrator level of the OAS (Level 3 described below).

Power Teacher In the past, users had requested that teachers obtain access to update student information and classes. In response, the OAS has added a new `Power Teacher' account. The `Power Teacher' has the ability to create or modify student accounts, create and modify classes, and view information on all classes in a school. The use of the `Power Teacher' allows schools and districts to change the process flow; permitting the administration of benchmarks and registration of students to be shared throughout the school, including grade-level and department chairs.
15

Georgia'sTesting Program

Level 3 The Georgia Online Assessment System now offers the ability to assign RESA or system-wide benchmarks through Level 3 Test Administration. Test coordinators retain their access to the Level 2 items (which teachers use to create their tests); but now, there is a more secure item bank that includes a level of items available only to RESA and system test coordinators (Level 3). The Level 3 items reflect those grades and subject areas that have transitioned to the GPS and are aligned to their respective GPS standard and element.

RESAs and systems may

obtain reports that depict

student

performance

aggregated to their level or `drilled down' from

RESAs to system to building to student. Level 3 Test

Administration also includes the Super Class a system

or RESA wide `class' of students that can be assigned

specific assessments very useful for RESA/district-

wide remediation programs.

Assessment for Learning

The OAS can be utilized for a variety of purposes, especially `Assessment for Learning'. Students can gain valuable experience in becoming self-assessors, learning to evaluate their own work and educational progress. Parents may see the kinds of questions their children will be asked to answer on the end-of-year assessments. This facilitates communication and parental monitoring of progress and provides an additional resource for parents who wish to reinforce or enhance learning at home. Teachers may create customized tests as they complete instructional units or sequences of instruction. These tests can then be administered to individual students and/or groups of students to diagnose strengths and

weaknesses and to provide opportunities for enrichment and remediation.
The GaDOE has introduced the Student Achievement Pyramid of Interventions, which is designed to help ALL Georgia students continue to make gains in achievement. The Pyramid of Interventions is a conceptual framework that illustrates layers of instructional efforts that can be provided to students according to their individual needs. Each of the four tiers of the Pyramid of Interventions includes a process of "progress monitoring" and successive interventions. The Georgia Online Assessment System works as one method of monitoring progress so teachers and administrators can more readily implement a standards-based classroom that is challenging and focused on each child's needs. For more information on the Pyramid of Interventions visit the School Improvement section of the GaDOE website.
The Georgia Online Assessment System truly works in conjunction with the `Assessment for Learning' model of instruction. It is critical that teachers monitor learning on a regular basis throughout the year in order to effectively implement interventions for enhancing student achievement. `Assessment for Learning' is a continuous process that includes Assessment, Learning, Interpretation, and Feedback. This cycle, implemented regularly throughout the year, becomes a part of the learning itself; students are always aware of what they are expected to accomplish and how successful they are on each step of the learning cycle. The GaDOE would like the Georgia Online Assessment System to be a support for teachers and learners, an easy-to-use, effective tool that facilitates student achievement.

16

Georgia'sTesting Program

Getting Started To gain access to the system, users should go to www.georgiaoas.org and use their Login ID and password to enter the system. Personal logins and passwords are necessary to access teacher-assigned tests and to share results obtained. These personal login IDs and passwords are assigned by school districts to the students and teachers at their schools. However, parents and others without personal login IDs can log on to the OAS using generic login IDs and passwords for each grade.

Generic logins to gain access to OAS Student Tests:

To see Student Tests for this grade: Use this login:

Use this password:

1st Grade

Grade1

Grade1

2nd Grade

Grade2

Grade2

3rd Grade

Grade3

Grade3

4th Grade

Grade4

Grade4

5th Grade

Grade5

Grade5

6th Grade

Grade6

Grade6

7th Grade

Grade7

Grade7

8th Grade

Grade8

Grade8

High School

GradeHS

GradeHS

In order to learn more about the system, the OAS has an information site at http://info.georgiaoas.org. On this site are numerous FAQs, manuals, slide shows, and QuickStart guides that provide everything from the basics to detailed overviews for experienced users. There are also training opportunities delivered throughout the state that are available for districts through a train-the-trainer model. The training, along with the online resources found at the information site, gives districts everything they need to redeliver training to their school staff.

Throughout this year, it is our hope that educators, parents, and students will use the OAS early, often, and effectively! What we assess, how we assess, and how we communicate the results sends a clear message to students about what is worth learning, how it should be learned, what elements of quality are most important, and how well we expect them to perform. We hope the OAS is a resource you will use throughout the year to complement your tremendous efforts in the classroom.
17

Georgia'sTesting Program

Georgia's Writing Assessment Program

Test Development Process for Writing Assessments 2005-2007

An essential skill for students and worthy of assessing

Focus Groups

Core Development Tea m
(Rubric Development)

Advisory Committee
(Prompt Writing)

Writing is a key component of the new Georgia Performance Standards. As schools and school systems have been transitioning to the English/Language Arts GPS, the Testing Division has been busy redeveloping the writing assessment program. Over the last year and a half, several committees of Georgia educators have examined the existing writing assessment program and made recommendations for the new writing assessments. The following provides a comprehensive overview of the writing assessments for Grades 3, 5, 8, and 11. In revising the writing program, the committees envisioned an assessment system that would evaluate a student's writing performance in each of the four grades; that is, in an analytical fashion on the same domains in order to provide longitudinal consistency. It should be noted that the new writing assessment for grade 11 will not become effective until fall 2007; the existing test will remain in place for the 2006-2007 school year.

Scoring of Field Test Papers

Benchmark Committee
(Range Finding)

Field Test Administration

Analysis of Field Test Data

Bias Review

Standard Testing

Operational Assessment (2007)
This section of the newsletter provides an in-depth view of the writing assessments for the 2006-2007 school year and provides highlights for the new GHSWT that will be implemented in the 2007-2008 school year. The chart on the last page summarizes the key features of each of the assessments.
Grade Three (3) Writing Assessment

Development Process of the New Writing Assessments
As with any test development in Georgia, the Testing Division works with groups of Georgia educators who are critical to the various steps of the test development process, including:
defining what knowledge and skills will be measured,
identifying what the student is expected to know and be able to do,
specifying the format of the test and test questions,
writing, reviewing, and refining writing prompts for field-testing, and
setting performance standards for students who Meet or Exceed standard.

Description The writing assessment for grade three consists of teacher evaluation of student writing using an analytic scoring system. The Grade 3 Assessment and Instructional Guide contains the scoring rubric; types of writing required by the GPS (narrative, informational, persuasive, and response to literature); good practices for the instruction of writing; sample student papers; and ways to evaluate student writing. Using representative samples of student writing, third-grade teachers are to use the analytic scoring rubrics in the Guide to determine the performance levels in each domain for each child in the classroom. Teachers collect writing samples by providing many opportunities for students to produce the various types of writing throughout the year.
Types of Writing The Georgia Grade Three Writing Assessment covers four types of writing: narrative, informational, persuasive, and response to literature.

18

Georgia'sTesting Program

1. Narrative Relating Personal Experience- Writing assignments should direct students to recount an event grounded in their own experiences. The assignment should elicit a story with a plot and characters rather than a list. Creating an Imaginative Story- Writing Assignments should direct students to produce stories that are grounded in imagination or fantasy.
2. Informational Writing Assignments may be related to all content areas specified in the Grade 3 GPS and may be produced during content area instruction. Writing assignments may be related to any type of non-fiction writing whose purpose is to inform or explain a topic to a reader. Students should incorporate information from resources (books, on-line sources, etc.) without copying the information verbatim. Paraphrasing information and using technical vocabulary from source material is appropriate for the informational assessment sample.
3. Persuasive The writing assignment should direct students to take a position on an issue or topic with which they are familiar. The assignment may occur after the class has researched the issue or read related texts. The assignment may be part of a lesson on the issue in a particular content area.
4. Response to Literature The assignment should direct students to form and support a position in response to a text they have read. The assignment should be linked to a specific piece of literature (short stories, biographies, fables, plays, poetry, and chapter books). Plot summaries or the retelling of an entire story are not appropriate responses to literature.

Analytic and Holistic Scoring The scoring system is analytic. Analytic scoring means that more than one feature or domain of a paper is evaluated. Each domain itself is scored holistically. The score assigned indicates the test rater's overall impression of the writer's command of the components using predetermined scoring criteria contained in the Scoring Rubrics. Accurate scoring requires balancing a writer's strengths and areas of challenge.
Student writing will be assessed analytically in four domains: Ideas, Organization, Style, and Conventions. Analytic scoring will provide detailed information on student writing, including performance levels.
Time Line Because writing assessment at grade three is an outgrowth of the writing instruction program, each elementary school or system should develop a plan at the beginning of the school year for teaching the four genres of writing and collecting assessment samples.
There is no "correct" order for teaching the writing genres in a school year.
Teachers may choose to teach all four genres of writing throughout the school year or teachers may choose to teach each genre at a particular time in the school year.
Some school systems integrate Response to Literature into the Reading Workshop curriculum and teach it throughout the school year simultaneously with the other three genres.
The steps of the writing process (prewriting, drafting, revising, editing, and publishing) should be taught throughout the year because third graders may not have fully learned this process and will need help applying each stage of the process to each genre of writing. For each genre, teachers should model the steps of the writing process, providing assistance and conferencing at each step.
Assessment samples should demonstrate what the student has learned to apply independently. During the course of effective instruction, the teacher may provide guidance and feedback that the students copy into their writing as they are learning to edit and revise. This type of assistance, while appropriate

19

Georgia'sTesting Program

for instruction in the writing process, is not appropriate for collecting assessment samples as it would not demonstrate writing the student is capable of producing independently. For the Grade 3 Writing Assessment samples, teachers must select one writing sample per genre for each student. After completing each genre unit, in which students practice the steps in the writing process, the teacher gives a writing assignment for the purpose of collecting the assessment sample. Because the writing performance levels for each grade 3 student will be passed on to the grade 4 teachers for instructional planning, it is essential that the assessment samples reflect what students can do independently. Using a writing prompt is an option for collecting assessment samples, but any classroom assignment that allows each student to demonstrate understanding of the writing process in that genre is appropriate.
During the last two weeks of March, teachers complete the Individual Student Report and the Summary Report. Writing samples may be included in the student's permanent record.

Scoring Procedures and Types of Scores For each student in the class, teachers should assemble multiple samples of writing that have been collected from a variety of classroom writing activities. At least one representative writing sample for each of the four types of writing should be collected. The scoring rubrics are to be applied with the same latitude and the same rigor to all four types. Teachers should rate the samples individually to determine the performance level that best represents a student's usual performance across a variety of writing tasks and types of writing. There are three performance levels represented: Does Not Meet, Meets, and Exceeds.

Reporting The classroom teacher completes the Individual Student Report form and the Summary Report identifying each student's performance level in each genre and domain. Individual student reportsone copy to be given to parent(s)/guardian(s) and one copy to be retained in the student's permanent recordare also included in reporting. The Summary Report should be sent to the GaDOE-designated test scoring and reporting contractor. School and system summary reports will be furnished to systems. The scoring rubrics and writing samples may be retained and forwarded to the grade four teachers.

20

Georgia'sTesting Program

New Writing Assessments Have Consistent Scoring Rules and Scale Scores in Grades 5, 8, and 11

Analytic and Holistic Scoring The scoring system is analytic. Analytic scoring means that more than one feature or domain of a paper is evaluated. Student writing will be assessed analytically in four domains: Ideas, Organization, Style, and Conventions. Analytic scoring will provide detailed information on student writing, including performance levels.

Each domain itself is scored holistically. Domains consist of several components. A component is a feature of writing within a particular domain. For example, "Focus" is a component of the Ideas domain. The score assigned indicates the test raters' overall impression of the writer's command of the components using predetermined scoring criteria contained in the Scoring Rubrics. Accurate scoring requires balancing a writer's strengths and areas of challenge.

The weight of each domain reflects the contribution of each domain to the student's total score. Weighting means that the score a rater assigns is multiplied by the weight (importance) assigned to the domain.

Each paper is scored by two raters. Raters who score the student compositions are trained to understand and use the standardized scoring system. The raters score each paper independently. Each of the four domains of effective writing is evaluated. Although these domains are interrelated during the writing process, a strength or area of challenge is scored only once under a particular domain. Scores in each domain range from 1 to 5 (5 being the highest score). The total weighted raw scores range from 10 (1's in all four domains) to 50 (5's in all four domains). These scores represent a continuum of writing that ranges from inadequate to very good. Points on the continuum are defined by the scoring rubric for each domain. Each score point itself represents a range of papers. Domain scores are combined to obtain a total score for each student. The following charts illustrate how the student papers are scored.

Domain Idea Organization Style Conventions Total Raw Score

Scoring for Grades 5, 8, and 11 Writing Assessments

Domain Weight

Point Range

Calculation of Weighted Score

40%

1 to 5 (2 x Rater No. 1's score) + (2 x Rater No. 2's score)

20%

1 to 5 (1 x Rater No. 1's score) + (1 x Rater No. 2's score)

20%

1 to 5 (1 x Rater No. 1's score) + (1 x Rater No. 2's score)

20%

1 to 5 (1 x Rater No. 1's score) + (1 x Rater No. 2's score)

Sum of above

The total raw score is then converted to a three-digit scaled score ranging from 100 to 350. A student's score is also reported in terms of three performance levels; does not meet, meets, and exceeds.

21

Georgia'sTesting Program

Performance Levels for Grades 5, 8, and 11 Writing Assessments

Performance Level

Scale Score Range

Does Not Meet

Below 200

Meets

200 to 249

Exceeds

250 or above

Grade Five (5) Writing Assessment
Description The writing assessment for grade five consists of an evaluation of each student response to an assigned prompt. Students are assigned a topic from a prompt bank representing three genres: narrative, informational, and persuasive. Students are allowed approximately 120 minutes to write their essays. The writing assessment must be administered in one day. A makeup assessment is also given the following day.
Types of Writing The Georgia Grade 5 Writing Assessment is a test of narrative, informational, and persuasive writing. Because topics will be spiraled, students may receive any one of the three writing topics thus requiring them to be prepared to write in informational, narrative, and persuasive genres. Topics will be released after each test administration and will become part of the practice topic bank.

Grade Eight (8) Writing Assessment
Description The writing assessment for grade 8 consists of an evaluation of each student response to an assigned prompt. Students are assigned a topic from a prompt bank representing two genres: expository and persuasive. Students are allowed approximately 100 minutes to write their essays. The writing assessment must be administered in one day. A makeup assessment is also given the following day.
Types of Writing The Georgia Grade 8 Writing Assessment is a test of expository and persuasive writing. Because topics will be spiraled, students may receive either an expository or persuasive writing topic, thus requiring them to be prepared to write in both genres. Topics will be released after each test administration and will become part of the practice topic bank.
Time Line The Grade 8 Writing Assessment is administered during the third week of January. All grade 8 students will take the assessment on the same day. There will also be one day for a make-up assessment. The testing time includes 100 minutes for student writing. Extra time will be allowed as specified in a student's Individual Education Plan (IEP), Section 504 Plan, or Test Participation Plan (TPP).

Time Line The Grade 5 Writing Assessment is administered during the first week of March. All grade five students will take the assessment on the same day. There will also be one day for a make-up assessment. Testing time includes 120 minutes for student writing. Each system will have the flexibility to determine what time of day to administer each of the sessions. Extra time will be allowed as specified in a student's Individual Education Plan (IEP), Section 504 Plan, or Test Participation Plan (TPP).

22

Georgia'sTesting Program

Georgia High School Writing Test (GHSWT)
Note: This information is for the current GHSWT for the 2006-2007 school year. The new GHSWT will not be operational until fall 2007.
Description Students in the eleventh grade participate in the Georgia High School Writing Test and must pass the GHSWT to earn a regular education diploma. Students are asked to produce a response to one on-demand persuasive writing prompt. The writing test requires students to produce a composition of no more than two pages on an assigned topic. Results of the GHSWT are used to identify students who may need additional instruction in academic content and skills considered essential for a high school diploma.
Types of Writing The GHSWT is a test of persuasive writing. In persuasion, the writer assumes a position on an issue and uses language to influence the reader. The purpose is to express a writer's opinion on a subject either explicitly or implicitly. Through the support provided, the writer presents a convincing point of view.
Analytic and Holistic Scoring The scoring system is analytic. Analytic scoring means that more than one feature or domain of a paper is evaluated. Each domain itself is scored holistically. The score assigned indicates the test raters' overall impression of the writer's command of the components, using predetermined scoring criteria contained in the Scoring Rubrics. Accurate scoring requires balancing a writer's strengths and weaknesses.
Time Line The two-hour test administration includes 90 minutes of student writing time. The test is administered three times a year (fall, spring, and summer) so that students have multiple opportunities to take the test before the end of grade 12. The main administration of the GHSWT takes place in the fall of the eleventh grade year. All assessments must be completed in one day. A make-up prompt is provided for those students not in attendance on the first day of testing. A make-up opportunity is not available during the summer retest.

Scoring Procedures and Types of Scores Four qualities of writing are evaluated in the GHSWT. Each of these qualities is referred to as a domain. Each paper is scored in four domains: Content/Organization, Style, Conventions, and Sentence Formation. The weight of each domain reflects the contribution of each domain to the student's total score. Weighting means that the score a rater assigns is multiplied by the weight (importance) assigned to the domain.
Each paper is scored by two raters. Raters who score the student compositions are trained to understand and use the standardized scoring system. The raters score each paper independently, and each of the four domains of effective writing is evaluated. These qualities or domains of effective writing should be present in a composition regardless of the topic on which it is written. Although these domains are interrelated during the writing process, a strength or area of challenge is scored only once under a particular domain.
Scores in each domain range from 1 to 4 (4 being the highest score). The total weighted raw scores range from 20 (1's in all four domains) to 80 (4's in all four domains). The scores represent a continuum of writing that ranges from inadequate to very good. Points on the continuum are defined by the scoring rubric for each domain. Each score point itself represents a range of papers. Scores assigned to each domain are summed. Domain scores are combined to obtain a total score for each student. In combining the domain scores, the Content/Organization score is given a weight of forty (40) percent, and the other domains of Style, Conventions, and Sentence Formation are given a weighted score of twenty (20) percent. The total score is then converted to a three-digit scaled score ranging from 400-600; 500 or higher is passing.
Student test reports also contain statements summarizing the student's performance on each of the four domains of effective writing. Strengths and areas of challenge in each domain are noted. This information can be used in planning remedial instruction for students who failed the test or in assisting any students who want to improve their writing.

23

Georgia'sTesting Program
Insight into the New GHSWT
The new writing assessment for high school will be implemented in fall 2007. This test will embody the same types of scoring changes that are taking place in grades 5 and 8 this year. There will be four domains: Ideas, Organization, Style, and Conventions. These domains will be rated on a scale of 1 to 5. Domain scores are combined to obtain a total score for each student.

Reporting for Grades 5, 8, and 11
There are several types of reports generated for each of the writing assessments in grades 5, 8, and 11. These include:
Student Label One label is provided for each student tested. The label is to be placed in the student's permanent school record. It contains the total scale score.
Student Report Two originals of the Student Report are provided; one is a student/parent copy, and one copy is for the permanent record and instructional use by the student's teacher(s).
Student Achievement Roster Two copies of Student Achievement Rosters are provided. School or class level rosters are provided depending on the choice made by the school. Rosters contain the names of all students tested. For each student, the roster displays the total writing score and a notation of the performance level.

Scale Score Rank Order Roster This roster lists students in rank order by scale scores. Two copies are provided.
School report A summary of student scores is provided for each school where testing was conducted. Three copies of the report are provided.
System Report For each system, a summary report is provided, which is identical in content to the school report. Three copies are provided.

24

Georgia'sTesting Program

Highlights of the Georgia Writing Assessments

New Grade 3 New Grade 5

2006-2007

2006-2007

New Grade 8 Current GHSWT New GHSWT

2006-2007

2006-2007

2007-2008

Format

Samples for each On-demand prompt genre collected genres spiraled in throughout year classroom

On-demand prompt genres spiraled in classroom

On-demand prompt

On-demand prompt

Genres or Types of Writing Time Line
Scheduled Dates
Domains and Weights Raters and Rating Process Scoring
Reporting

Narrative, Informational, Persuasive, Response to Literature

Narrative, Informational, Persuasive

Expository, Persuasive

Collected

120 minutes in one 100 minutes in one

throughout year day *

day *

March 19-30, 2007 (Rating window)

March 7, 2007 (Main); March 8, 2007 (Make-up)

January 17, 2007 (Main); January 18, 2007 (Make-up)

Ideas, Organization, Style, Conventions, (Domains are not weighted.)

Ideas (40%), Organization (20%), Style (20%), Conventions (20%)

Ideas (40%), Organization (20%), Style (20%), Conventions (20%)

Teachers rate each genre for each student.

Trained raters (2) per paper

Trained raters (2) per paper

Each domain is rated as Does Not Meet, Meets, or Exceeds.

A score of 1 to 5 is assigned to each domain by each rater.

A score of 1 to 5 is assigned to each domain by each rater.

Ratings are provided on each domain for each of the four genres. No composite score is given.

Scales scores range from 100 to 350: Does Not Meet (scores below 200), Meets (scores from 200 to 249), and Exceeds (scores of 250 or above).

Scales scores range from 100 to 350: Does Not Meet (scores below 200), Meets (scores from 200 to 249), and Exceeds (scores of 250 or above).

Persuasive
120 minutes in one day 90 minutes of writing time * September 27, 2006 (Main); September 28, 2006 (Make-up)
Retest Dates: February 28, 2007 March 1, 2007 July 11, 2007 Content/Organization (40%), Style (20%), Conventions (20%), Sentence Formation (20%)
Trained raters (2) per paper
A score of 1 to 4 is assigned to each domain by each rater.
Scale scores range from 400-600 with 500 or higher as passing.

Persuasive
120 minutes in one day 100 minutes of writing time *
September 26, 2007 (Main); September 27, 2007 (Make-up)
Retest Dates: February 27, 2008 February 28, 2008 July 2008
Ideas (40%), Organization (20%), Style (20%), Conventions (20%)
Trained raters (2) per paper
A score of 1 to 5 is assigned to each domain by each rater. Scales scores range from 100 to 350: Does Not Meet (scores below 200), Meets (scores from 200 to 249), and Exceeds (scores of 250 or above); 200 or higher is passing.

* Extra time is allowed as specified in a student's Individual Education Plan (IEP), Section 504 Plan, or Test Participation Plan (TPP).

25

Georgia'sTesting Program

The New Georgia Alternate Assessment (GAA)

The Georgia Alternate Assessment (GAA) has been redesigned to meet the mandates of the No Child Left Behind Act (NCLB) of 2001 and the Individuals with Disabilities Education Act (IDEA) of 2004. Both NCLB and IDEA require that all students, including students with significant cognitive disabilities, have access to a general curriculum that encompasses challenging academic standards. Additionally, states must ensure that all students are assessed for their progress toward meeting academic standards. The 2006 2007 school year marks the first operational administration of the new GAA.
How was the new GAA developed? Committees of Georgia educators led the redevelopment of the GAA. Committee meetings were held in January, February, May, July, and October of 2006. Committee members, the majority of whom teach students with significant cognitive disabilities, informed the design of the assessment, including the requirements of the portfolio, the scoring rubric, and general administration procedures and guidelines. The redesigned GAA was piloted in spring 2006 with 47 school systems throughout the state. Approximately 400 students participated in the assessment. For each student, a teacher compiled a portfolio in one assigned content area English/language arts, mathematics, social studies, or science. The pilot provided invaluable information about the assessment process and was used by the committee to make final recommendations for the operational administration.
Who takes the GAA? The GAA is designed for students with the most significant cognitive disabilities. Each student's Individualized Education Program (IEP) team determines how the student will participate in the state's assessment program. All special education students must participate in the state's assessment program in one of three ways:
in the general assessment program without accommodations;

in the general assessment program with accommodations; or
in the GAA.
The GAA may be considered for students with the most significant cognitive disabilities. The GAA is appropriate only when the IEP team determines that a student's cognitive disabilities prevent him/her from attaining grade-level achievement even with the very best instruction and when the IEP team is unable to identify appropriate accommodations that would allow meaningful participation in the general assessment program. The Georgia Department of Education (GaDOE) provides guidance for IEP team consideration in making appropriate assessment decisions. The guidance for GAA consideration was informed by the committee of Georgia educators. The GAA Participation Flowchart is posted on the GAA webpage, found at www.gadoe.org.
What does the new GAA involve? The GAA is a portfolio assessment. Portfolios are a compilation of work documenting, measuring, and reflecting student performance and progress on standards-based knowledge and skills over time. The portfolio may contain student work products, video clips of student performance, audiotapes of student responses, examples of student performance on paper-and-pencil tasks, and other information about student progress. The portfolio serves as a database for documenting achievement on the curriculum standards on which the student is assessed. A portfolio provides ongoing documentation of student skills, merges instructional and assessment activities, and allows each student to demonstrate progress toward grade-level standards.
Students eligible to participate in the GAA in grades K2 must be assessed in English/language arts and mathematics. Students eligible to participate in the GAA in grades 38 and 11 must be assessed in English/ language arts, mathematics, social studies, and science.

26

Georgia'sTesting Program
27

Georgia'sTesting Program

All students must be assessed on two standards in English/language arts and mathematics and on one standard in social studies and science (paired with a Characteristics of Science standard in grades 3 - 7). The GAA Blueprint, provided on the GAA website and in the GAA Examiner's Manual, provides the required strands, standards, and choices available for each grade and content area.
The instructional tasks and, therefore, the student evidence submitted in the portfolio must be clearly aligned to the state-mandated curriculum standards on which the student is being assessed, whether from the Georgia Performance Standards (GPS) or the Quality Core Curriculum (QCC). These tasks may follow alternate achievement standards (expectations that are less complex or addressing access or entry concepts and skills, including pre-requisite skills). However, such alternate standards must still be challenging for the student and must be focused on grade-level standards. Teachers of students participating in the GAA are strongly encouraged to discuss the alternate assessment and alignment of instructional activities with their general education colleagues. General educators are a valuable resource for suggesting instructional activities that are aligned to the curriculum standards.
The GAA requires teachers to collect evidence of student learning and progress during two collection periods. The first collection period provides evidence of the student's entry-level performance (initial/baseline performance of the skill); the second collection period provides evidence of the student's achievement/ progress to date. There must be a minimum of three weeks between collection periods.
Two types of evidence will be required for submission for each entry and for each collection period: primary evidence and secondary evidence (a total of 4 pieces of evidence per entry). Primary evidence is work that has been produced by the student. Secondary evidence documents, relates, charts, or interprets the student's performance on instructional tasks. The secondary evidence must be based on tasks or activities that are related to but are different than those shown in the primary evidence.

How will the GAA be scored? The GAA portfolio entries will be scored on four discrete dimensions: Fidelity to Standard, Context, Achievement/Progress, and Generalization. A separate score will be assigned for each dimension. The following is a brief description of these dimensions:
Fidelity to Standard assesses the degree to which the student's work addresses the gradelevel standard to which it is aligned;
Context assesses the degree to which the student work exhibits the use of grade-appropriate materials in a purposeful and natural/ real-world application;
Achievement/Progress assesses the increase in the student's proficiency of skill across the two collection periods; and
Generalization assesses the student's opportunity to apply the learned skill in other settings and with various individuals in addition to the teacher or paraprofessional
The GAA will be scored by the GAA contractor, Questar Educational Systems. Questar has partnered with GaDOE in the development of the new GAA and has worked with the committees of Georgia educators to ensure the assessment meets the needs of the state as well as the students. Questar has experience with scoring alternate assessment and has worked with other states, including Virginia, Ohio, and Arkansas.
Georgia educators will advise Questar on scoring procedures through an activity called range finding. In early April 2007, a committee of Georgia educators will be convened to score a sample of portfolios representing all grade levels and content areas. Through this activity, Georgia educators will set the parameters for each score point (the upper and lower limits for each point on each rubric dimension). The committee will discuss each entry and provide a rationale for why they assigned the portfolio each score.
From range finding, scoring guides will be created to train Questar's scorers. The scorers will be trained using a rationale provided by Georgia educators. After training, the scorers must demonstrate they can apply

28

Georgia'sTesting Program

the rubric appropriately by qualifying for scoring. This means the scorers will rate a set of portfolios and must score them exactly as the Georgia educators did. Scorers who cannot qualify are not allowed to score.
In addition to ensuring scorers are trained and meet qualifications, the scoring process itself is carefully checked to maintain a high level of reliability. Periodically, portfolios with confirmed scores are placed in the mix to ensure scorers continue to apply the rubric as Georgia educators intended. Additionally, some portfolios will be scored twice to ensure consistency in scoring. GaDOE staff from both the Testing Division and the Division of Exceptional Students will be on site to monitor scoring.
The next step in the process, as is true with any newly developed test, will be standard setting. Again, a committee of Georgia educators will be convened to make recommendations about portfolios that do not meet expectations, meet expectations, or exceed expectations. Because the primary purpose of the GAA is to assess student progress toward grade level academic standards, the GAA development committee recommended three stages of progress (or performance levels) describing student performance Emerging Progress, Established Progress, and Extending Progress.
Standards-setting panels (comprised of both Georgia special education teachers and general classroom educators) will make recommendations for each content area by grade band, (K-2, 3-5, 6-8, and 11). Score reports will provide information on the stage of progress achieved by each student in each content area as well as provide the assigned scores for each rubric dimension. Reports will be provided at the student, school, system, and state levels.
How will the GAA be used in AYP? Importantly, GAA scores will contribute to AYP determinations. For AYP purposes, Emerging Progress will be considered Basic, Established Progress will be considered Proficient, and Extending Progress will be considered Advanced.

to one percent. In other words, the number of proficient scores included in AYP calculations based on the GAA may not exceed one percent of all students enrolled in the tested grades unless a school system has requested and received a waiver from the GaDOE. This one percent cap does not apply to individual schools but does apply at the system and state levels.
What type of support is available to teachers of students with significant cognitive disabilities? In spring 2005, the Division of Exceptional Students held regional focus groups across the state to solicit input on curriculum access for students with significant cognitive disabilities. These focus groups informed the development of the Curriculum Resource Guide, outlining the four-step process through which instruction is aligned to grade-level academic standards.
During the 2005-2006 school year, the Division of Exceptional Students hired two teachers who had extensive experience in working with students with severe cognitive disabilities at all levels and placed them on special assignment to work with districts in making the curriculum accessible to these students. They provided training on curriculum access for over 1,800 educators. Additionally, a cadre of teachers was identified for more intensive training. These Core Access Teachers (CATs) serve as regional experts and have helped to design instructional tasks that align to the state curriculum. A wealth of information on making the GPS accessible to students with significant cognitive disabilities is provided at the Georgia standards website at www.georgiastanadards.org.
During the 2006-2007 school year, the Division of Exceptional Students continues to work with the CATs and, in addition, has established a cadre of Communities of Practice (COPs) teachers. Each school system in the state was asked to identify COPs candidates to receive training in best practices to take back to their school systems throughout the year. These COPs teachers receive periodic training and updates throughout the year and serve as in-district resources.

Federal regulations limit the percentage of GAA scores that may be included as proficient in AYP determinations

29

Georgia'sTesting Program

Finally, a message board has been established to share ideas about instructional tasks and to facilitate collaboration with other educators across the state. Access to the message board requires a password and user ID, which can be obtained by contacting staff at the Division of Exceptional Students at (404) 656-3963.

What resources are available for the GAA? Several resources are available to local systems on the GaDOE website for training to administer the Georgia Alternate Assessment. These resources appear as links and can be found at www.gadoe.org/ci_testing.aspx in the portlet labeled GAA Resources. The resources include the following documents:

Why did Georgia decide to use the portfolio format for its alternate assessment? Both the Testing Division and the Division of Exceptional Students spent considerable time reviewing other states' alternate assessments. The portfolio format was ultimately chosen because it provides the flexibility necessary to ensure this small yet very diverse group of students receives instructional services that are meaningful and purposeful.

Examiner's Manual Coordinator's Manual GAA Portfolio Resource Manual GAA Blueprint Portfolio Forms (electronic)

The portfolio allows the teacher to choose, within certain parameters, which academic standards will be assessed. Portfolios respect the teacher's professional judgment and knowledge about what is best for each individual student. Other alternate assessment formats, such as performance tasks and rating scales, are more standardized with all students completing the same activities. The portfolio offers more flexibility and individualization.

Parent Brochure Rubric Participation Flowchart

Without a doubt, the compilation of portfolio entries is very time-consuming for the teacher. Clear, detailed documentation describing student work is required. However, the advantage of designing individualized instruction and assessment for each student is considered critically important. Preliminary research on different types of alternate assessments has indicated that portfolios enhanced instructional opportunities over and above those experienced by students assessed by other types of alternate assessments.

30

Georgia'sTesting Program
Notes
31

Georgia Department of Education Testing Division
1554 Twin Towers East Atlanta, Georgia 30334
Tel (404) 656-2668 Tel (800) 634-4106 Fax (404) 656-5976
www.gadoe.org
Federal law prohibits discrimination on the basis of race, color, national origin, gender, or disability, in educational programs or activities receiving federal financial assistance. Employees, students, and the general public are hereby notified that the Georgia Department of Education does not discriminate in any educational programs or activities or in employment policies. Inquiries regarding the application of these practices may be addressed to the Georgia Department of Education, Twin Towers East, Atlanta, Georgia 30334, (404) 656-2800.