What do we mean by “equity” in assessment? One significant aspect of equity is the absence of test bias. Here are a few key ideas about avoiding bias that district and local test makers can apply.
A brief history of equity in education

Equity in educational opportunity has been a major focus of federal legislation and programs for over a half a century. It started with the passing of the Elementary and Secondary Education Act in 1965 and continued in all its subsequent reauthorizations, including No Child Left Behind and the Every Student Succeeds Act. Separate legislation for students with disabilities was passed in 1975 as the Education for All Handicapped Children Act, which became the Individuals with Disabilities Education Act of 1990. These laws have had significant impact on educational testing at the state and local levels.

There are, of course, long lists of approved test accommodations and modifications that serve to “level the playing field” for students with disabilities who need particular supports to complete tests. The rule for state tests is that they should allow the same supports that these students have available to them during regular local instruction and testing.

Despite legislated advances in accessibility and available support, equity and bias in testing remain frequently misunderstood. A few critical elements can help school and district leaders identify, understand and address testing bias prior to and after administration.

How states avoid bias in tests

State testing programs apply two techniques to guard against producing items or tests that are biased against subgroups of students:

  1. Looking for direct statistical evidence of bias after item data are available
  2. Identifying and eliminating potential causes of bias early, during item writing

Educators naturally think about reliability and content validity, but may not consider bias and sensitivity.

How to understand bias

Perhaps surprisingly, tests that produce different results for different subgroups of students are not necessarily biased. Real achievement differences among subgroups exist, and a test that doesn’t detect them would be a poor test—not necessarily a biased one.

Sensitivity review

Another aspect of review during development is to look for ideas or topics that may upset individual students. For example, violence, death, and divorce are often included in lengthy lists of topics to avoid because students may have had recent experiences with such an event and may have emotional reactions. While it may seem unlikely that test questions would include such topics, if you are using authentic texts for reading assessments, they might indeed touch on serious and sensitive issues that could upset or significantly distract a test taker. Your reviewers can help identify potentially controversial or sensitive topics.

How to look for bias in test results

Large-scale testing programs conduct extensive analysis on test results for a number of reasons, including searching for potential bias. While after-the-fact statistical analyses of item bias may be more difficult to apply at a local level, district testing may involve large enough samples of students that some basic approaches may be applied.

For example, your district assessment team may be able to rank order test items by difficulty, separately by subgroups (item difficulty is measured by percent of correct answers or percent of possible points earned). If an item appears in a very different place in the rank orderings for the different groups, that can be a warning sign of potential item bias. Then, you and your team can examine the item for some of the causes of bias identified in bias and sensitivity guidelines. You can either revise the item or eliminate it in subsequent administrations of the test.

Beyond the test content

Another aspect of equity in testing has nothing to do with the test content, and that’s equity of access to technology. Students who are less experienced with computers and other devices may be hindered if they are required to complete tests online. Their results could suffer accordingly. With many states now requiring online testing for the general population of students, this can create testing inequity—until all students have adequate online experience. If a “digital divide” exists in your school or district, you’re already well aware of its effects. You probably can’t design a test that eliminates the problem, but you can certainly be aware of it when creating your own tests.

State test-development processes involve more formal approaches for detecting unfairness than most local test-creation efforts can support. However, district leaders and local educators can also do their parts to ensure their tests do not unfairly disadvantage a particular subgroup of students.

Stuart Kahl, P.h.D.
Dr. Stuart Kahl, founding principal and former CEO and president of Measured Progress, recently retired from full-time service to the organization. With more than 35 years of experience in large-scale assessment, Dr. Kahl is a frequent speaker at industry conferences and serves as a technical consultant to education agencies. He regularly publishes commentaries on current issues and trends in education. His recent interests include assessment literacy and curriculum-embedded performance assessment.

© Cognia Inc.

This article may be republished or reproduced in accordance with The Source Copyright Policy.

The information in this article is given to the reader with the understanding that neither the author nor Cognia is in engaged in rendering any legal or business advice to the user or general public. The views, thoughts, and opinions expressed in this article belong solely to the author(s), and do not necessarily reflect the official policy or position of Cognia, the author’s employer, organization, or other group or individual.