Item Analysis

What is an Item Analysis?
Item analysis examines student responses to individual test items or test questions in order to determine the quality of those items. It is valuable in improving test items which can be used again in later tests. It can be also used to eliminate misleading test items in a single test administration. Moreover, it will enhance instructors' skills in test construction, and identifying specific areas that need greater emphasis.

Classical Item Analysis Statistics
The following are item analysis statistics that you can use:

• reliability - determines the consistency of the test (test level statistics)
• difficulty - can be inferred from the number of students who answered the item correctly (item level statistics)
• discrimination - can be inferred from the number of respondents in the lower group who got the correct answer on the test item (item level statistics)

How to Item Analyze?
1. Score the test papers to obtain the total scores of the students
2. Arrange test scores of the students from highest to lowest
3. Split the test papers into a high group and low group

• For a class of ≤ 30 do a 50-50 split, For example, if your N=20, the upper 10 represent your upper group (nU) and the lower 10 represent your lower group (nL)
• For a large group, you may take the upper (nU) 25-27% and lower (nL) 25-27% (Nunnally, 1972; Wiersma & Jurs, 1990). For example: If there are 100 pupils taking the test, the nU (27%) = 27 pupils and nL (27%) = 27 pupils

4. Compute the index of difficulty by calculating the p-value (p) which is the proportion of students answering the item correctly.

p=R/T

where: p is the p-value; R is the total number of students answering the item right, and T is the total number of students answering the item

or you can use this to calculate:

D= [(U/nU+L/nL))]/2

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

5. Compute the discrimination index. Obtain first the p-value for the upper and lower group, then get the difference between the p-values. 

p=(Rᵤ/Tᵤ)-(Rₗ/Tₗ)

where: p is the p-value; Rᵤ is the total number of students answering the item right in the upper group; Tᵤ is the total number of students answering the item in the upper group; Rₗ is the total number of students answering the item right in the lower group; Tₗ is the total number of students answering the item in the lower group

or you can use this to calculate:

ID= (U/nU)-(L/nL)

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

Example: (using 25%)

The correct answer to a test item in a Science test is “b”. There were forty pupils in the class. The upper 25% consists of the top 10 pupils and the lower 25% consists of the lowest 10 pupils. Eight pupils from the upper group got the item correctly, and only 3 in the lower group got the correct answer. What is the index of difficulty and discrimination index of this test item? What action can be taken based on the results of D & ID? 

Solution:

D=[(U/nU+L/nL)]/2

D=[(8/10+3/10)]/2

D=(.8+.3)/2

D=1.1/2

D=0.55 (moderately difficult item)

ID= (U/nU)-(L/nL)
ID=(8/10)-(3/10)
ID=.8-.3
ID=0.50 (discriminating item)

5. Interpret the result of D and ID. To do this, you may use the difficulty & discrimination index table by Hopkins & Antes (1990) or the table below:

Using the table above, D=.55 means that the item is of moderate difficulty and ID=.50 means it is discriminating item. Thus, the action (moderately difficult🠖discriminating) is "include" the item. 

References:

1. Santos, R. G. (2007). Assessment of Learning 1. Quezon City: Lorimar 
2. Padua, R.N. and Santos, R. G. (1997). Educational Evaluation and Measurement: Theory, Practice, and Application. KATHA Publishing: QC.