A new method for detecting differential item functioning in. The textbook is focused on item response theory overall, but discusses differential item functioning and item bias. X fits an item response model when x are item scores e. Differential item functioning analysis with ordinal logistic regression techniques difdetect and difwithpar paul k. Differential item functioning request pdf researchgate. Introduction essentially the same differential item functioning dif procedures have been used in practice since the late 1980s. The methods were applied to simulated data sets of varying test length 20, 40, and 60 items and sample size 200, 400, and 600 examinees for both equal and unequal underlying ability between groups as well as for both fixed and. Differential item functioning dif is a statistical characteristic of an item that shows the extent to which the item might be measuring different abilities for members of separate subgroups. A handbook on the theory and methods of differential item functioning dif.
Potential sources of differential item functioning in the. Differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. An introduction to differential item functioning analysis eric. It is also germane to consider how the individual item level dif indices aggregate across all n items of the test, referred to as differential test functioning dtf. If the differential item functioning is a result of previously unattributed group differences, for example differences in real abilities, then the test item is often maintained in the testing instrument. An item displays dif when test takers possessing the same amount of an ability or trait, but belonging to different subgroups, do not share the same likelihood of correctly answering the item. Differential item functioning analysis with ordinal. Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. Thus, differentially functioning items elicit different. It is also germane to consider how the individual itemlevel dif indices aggregate across all n items of the test, referred to as differential test functioning dtf.
Bechger and maris proposed using a differential item pair functioning dif test, which focuses on comparing item pairs instead of seeing dif as an item property. In addition, we urge researchers to combine statistical analysis with context. Phd doctor of philosophy thesis, university of iowa, 2011. Differential item functioning related to age in the. The assessment of differential item functioning dif has become an integral part of determining the validity of standardized tests and multiitem surveys. To obtain a pdf or a print copy of a report, please visit. Differential item functioning analysis with ordinal logistic. Current problems and future directions hossein karami, university of tehran, iran mohammad ali salmani nodoushan, iecf, iran. Potential sources of differential item functioning in the adaptation of tests paula elosua and alicia lopezjauregui department of psychology university of the basque country, spain this report shows a classification of differential item functioning dif sources that have an effect on the adaptation of tests. Assessment developers design and construct questionnaires or tests including sets of items that measure, for example, cognition, personality traits, or political views. Teresi, 1, 2 katja ocepekwelikson, 2 marjorie kleinman, 1 joseph p.
Differential item functioning magnitude and impact measures. Differential item functioning dif of sf12 and qlesqsf. Pdf differential item functioning dif is a key component in the evaluation of the fairness and validity of educational and psychological tests. Nonetheless, if the differential item functioning is a result of language choices that provide an advantage to one subgroup over another, or. Dorans kyndra middleton august 2008 ets rr0843 research report. Sep 01, 2017 this is the webpage for the handbook on differential item functioning. Differential item functioning dif in composite health measurement.
Differential item functioning procedures for polytomous items. We provide a tutorial on differential item functioning dif analysis, an analytic. As demonstration, a dataset from a 40item math test in a statewide testing program is analyzed by two widely. The methods were applied to simulated data sets of varying test length 20, 40, and 60 items and sample size 200, 400, and 600 examinees for both equal and unequal underlying ability between groups as well as for both fixed and varying item. The aims of this paper are to present findings related to differential item functioning dif in the patient reported outcome measurement information system promis depression item bank, and to discuss potential threats to the validity of results from studies of dif. Paper 29002015 multiple ways to detect differential item. This new edition presents an uptodate description of dif. Recommendations for conducting differential item functioning. Investigation of for the of differential item functioning. The rows in each group refer to the levels from lower to higher, with the fourth row indicating the sum of each ability level. Item response theory detects differential item functioning. Logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores.
Differential item functioning procedures for polytomous items when examinee sample sizes are small. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods, an r. Differential item functioning, dichotomous test, magnitude.
Current issues 951 people who answered the item correctly at the ability level m, and the proportion of people who answered the item correctly at the ability level m, respectively. Request pdf differential item functioning about the authors series editors introduction 1. Differential item functioning in pisa due to mode effects. Psychological testing, differential item functioning dif is considered as validity evidence. Differential item functioning dif exists when examinees of equal ability from different groups have. This article provides a brief primer overview of differential item functioning dif analysis. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the same way for all subgroups. A handbook on the theory and methods of differential item functioning dif logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores bruno d. Investigating linguistic sources of differential item. A general framework and an r package for the detection of. Detecting differential item functioning dif in standardized multiplechoice test. Judicious application of this methodology by the researchers, however, requires an. The difference with traditional procedures is that dif is defined in terms of the relative difficulties of pairs of itemswhich are identified from the observationsand not in terms.
The use of new item formats, new item types, and new test. Number of items by gender differential item functioning category, 2017 18 80 celdt 201718 edition technical report california department of education november 2018 1. We present an ordinal logistic regression model for identi. Differential item functioning dif occurs when items that are intended to measure a latent trait are unfair, favoring one group of individuals over another. Longitudinal differential item functioning detection using bifactor models and the wald test by mian wang a dissertation submitted to the graduate degree program in the department of psychology and the graduate faculty of the university of kansas in partial fulfillment of the requirements for the degree of doctor of philosophy. Effect of differential item functioning on test equating eric. Evaluation of two types of differential item functioning in. See the following manual entries for details about the individual dif tests, including syntax and worked. Relatively fewer studies examined an itemlevel approach to measurement equivalence, particularly in settings where a large number of groups is included. Differential item functioning dif is investigated to ensure that each item displays a consistent pattern of responses irrespective of the characteristics of the respondents.
Measurement invariance and differential item functioning. Differential item functioning and health assessment. I hope that you learn something new and interesting about differential item functioning. Pdf empirical comparison of three methods for detecting. Differential item functioning dif of sf12 and qlesq. Analysis of differential item functioning in the depression. If the factor bringing about such a difference is not part of the construct of focus in the test, then the test would be biased.
Differential item functioning dif has been widely used in healthcare, business management, and educational measurement. Item response theory detects differential item functioning between healthy and ill children in qol measures. With the rising concerns over the fairness of language tests, differential item functioning dif has been increasingly applied in bias analysis. Differential item functioning dif is the preferred psychometric term for what is otherwise known as item bias.
Differential item functioning responsive translation. Gibbons, phd, lance jolley, ms, and gerald van belle, phd introduction. Differential item functioning dif is an important issue of interest in psychometrics and educational. A new method for detecting differential item functioning. Assessing dif helps to understand the nature of instruments, to assess the quality of a measure and to interpret results. In the context of tests, dif occurs when people from different groups with the same ability have systematically different responses to speci. The purpose of the present analysis is to use differential item functioning dif to identify differences in the performance of native and immigrant students in pisa 2009 that can be directly related to their responses to particular items. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods. Grm item fit, measurement invariance absence of differential item functioning dif due to age, gender, center, duration, and location of complaints and precision.
View differential item functioning research papers on academia. The interest toward multilevel models increased with the rasch model, which. Package difr may, 2020 type package title collection of methods to detect dichotomous differential item functioning dif version 5. Combine items exhibiting dif with other items of the scale n3. Pdf an introduction to differential item functioning. Detecting differential item functioning using generalized. This is the webpage for the handbook on differential item functioning. Differential item functioning research papers academia. Table 30 supports the investigation of item bias, differential item functioning dif, i.
Differential item functioning dif occurs when examinees with the same ability have different probabilities of endorsing an item. The methods described thus far have addressed the concept of fairness and bias at the item level. Item response theory detects differential item functioning between healthy and ill children in qol measures michelle m. We analyzed 95 cognitive reading items, administered to students in 29 european countries. Differential item functioning procedures for polytomous. When such differences are found, the items are said to be functioning differentially. Differential item functioning dif refers to group differences in performance on a test item that cannot be explained by group differences in the construct targeted. Differential item functioning magnitude and impact.
This phenomenon is termed differential item functioning dif. Central to achieving comparability is to examine the presence of dif, and if dif is found, to investigate its sources to ensure differentially functioning items do not lead to bias. An application of item response theory irt using three parameter logistic model by ling siew eng lau sie hoe universiti teknologi mara sarawak abstract multiplechoice tests are the most common format used in standardized test for measuring cognitive ability. Does differential item functioning occur across respondents. Longitudinal differential item functioning detection using. Differential item functioning dif detection rates were examined for the logistic regression and analysis of variance anova dif detection methods. Differential test functioning and dif effect variance. Differential item functioning sage research methods. In this study, sources of dif were examined using thinkaloud protocols. Differential item functioning dif analysis is a key component in the evaluation of the fairness and validity of educational tests. Evaluation of two types of differential item functioning. This simulation study examines itemlevel differential item functioning dif in the context of international largescale assessment ilsa using a generalized logistic regression approach.
Gender differential item functioning on a national field. Differential item functioning dif is an important issue of interest in psychometrics and educational measurement. Figure 1 displays a scatterplot for the males and females item difficulties. Dif analysis investigates a differential characteristic of a test item between subpopulations of examinees and is useful in detecting possibly biased items toward a particular subpopulation. This study aimed to examine whether the items of the quality of life enjoyment and satisfaction. Differential item functioning procedures for polytomous items when examinee sample sizes are small by. A comparison of two differential item functioning detection. Differential item functioning columbia university mailman. Analysis of differential item functioning in the depression item bank from the patient reported outcome measurement information system promis.
774 851 992 965 808 625 826 155 12 965 271 450 108 1381 838 316 878 1401 1370 129 48 769 1442 645 662 340 473 1267