Item Analysis

What is an Item Analysis?
Item analysis examines student responses to individual test items or test questions in order to determine the quality of those items. It is valuable in improving test items which can be used again in later tests. It can be also used to eliminate misleading test items in a single test administration. Moreover, it will enhance instructors' skills in test construction, and identifying specific areas that need greater emphasis.

Classical Item Analysis Statistics
The following are item analysis statistics that you can use:

• reliability - determines the consistency of the test (test level statistics)
• difficulty - can be inferred from the number of students who answered the item correctly (item level statistics)
• discrimination - can be inferred from the number of respondents in the lower group who got the correct answer on the test item (item level statistics)

How to Item Analyze?
1. Score the test papers to obtain the total scores of the students
2. Arrange test scores of the students from highest to lowest
3. Split the test papers into a high group and low group

• For a class of ≤ 30 do a 50-50 split, For example, if your N=20, the upper 10 represent your upper group (nU) and the lower 10 represent your lower group (nL)
• For a large group, you may take the upper (nU) 25-27% and lower (nL) 25-27% (Nunnally, 1972; Wiersma & Jurs, 1990). For example: If there are 100 pupils taking the test, the nU (27%) = 27 pupils and nL (27%) = 27 pupils

4. Compute the index of difficulty by calculating the p-value (p) which is the proportion of students answering the item correctly.

p=R/T

where: p is the p-value; R is the total number of students answering the item right, and T is the total number of students answering the item

or you can use this to calculate:

D= [(U/nU+L/nL))]/2

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

5. Compute the discrimination index. Obtain first the p-value for the upper and lower group, then get the difference between the p-values. 

p=(Rᵤ/Tᵤ)-(Rₗ/Tₗ)

where: p is the p-value; Rᵤ is the total number of students answering the item right in the upper group; Tᵤ is the total number of students answering the item in the upper group; Rₗ is the total number of students answering the item right in the lower group; Tₗ is the total number of students answering the item in the lower group

or you can use this to calculate:

ID= (U/nU)-(L/nL)

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

Example: (using 25%)

The correct answer to a test item in a Science test is “b”. There were forty pupils in the class. The upper 25% consists of the top 10 pupils and the lower 25% consists of the lowest 10 pupils. Eight pupils from the upper group got the item correctly, and only 3 in the lower group got the correct answer. What is the index of difficulty and discrimination index of this test item? What action can be taken based on the results of D & ID? 

Solution:

D=[(U/nU+L/nL)]/2

D=[(8/10+3/10)]/2

D=(.8+.3)/2

D=1.1/2

D=0.55 (moderately difficult item)

ID= (U/nU)-(L/nL)
ID=(8/10)-(3/10)
ID=.8-.3
ID=0.50 (discriminating item)

5. Interpret the result of D and ID. To do this, you may use the difficulty & discrimination index table by Hopkins & Antes (1990) or the table below:

Using the table above, D=.55 means that the item is of moderate difficulty and ID=.50 means it is discriminating item. Thus, the action (moderately difficult🠖discriminating) is "include" the item. 

References:

1. Santos, R. G. (2007). Assessment of Learning 1. Quezon City: Lorimar 
2. Padua, R.N. and Santos, R. G. (1997). Educational Evaluation and Measurement: Theory, Practice, and Application. KATHA Publishing: QC.

Internal Consistency Reliability: Split Half Method

Internal Consistency Reliability is a subdivision of the test into two or more equivalent parts.  In internal consistency reliability, we judge the reliability of the instrument by estimating how well the items that reflect the same construct yield similar results. 

One method we can use is split-half. In split-half reliability, we randomly divide all items that purport to measure the same construct into two sets. How? 

First, divide the test into half usually using the odd-even technique. As shown in the illustration below, the items were divided into odd-even numbered items, where odd-numbered items (1, 3, 5) forms the "x" group while even-numbered items (2, 4, 6) forms the "y" group.

Second, find the correlation of scores using Pearson r formula. To do this, make a table as shown in the example below.
Example: Four pupils took a 50-item test. Below are the results

The table above shows that odd-numbered items are denoted by (x) while even-numbered items are denoted by (y). The rows in the two columns show the scores of the four (4) pupils who took the 50-item test. For example, pupil 1 scored a total of 30 out of 50 in the test. When divided into odd-even numbered items, it shows odd=10 and even=20. Likewise, pupil 2 scored only 13 out of 50 in the test where odd-numbered items=5 and even-numbered items=8.

Moreover, the first two columns are given based on the result of the test while the last three columns need to be solved. For the third and fourth columns, all you have to do is to square the x and y variables. The last column is the product of x and y. 

Finally, the last row provides the sum of all the columns. Once you have all this data you can now substitute the Pearson r equation. Just follow the example below.

Third, adjust & re-evaluate correlation using Spearman-Brown formula

The purpose of re-evaluating the correlation is to determine the reliability of the test as a whole. Since we divide the test into half, the result of the correlation in step 2 using Pearson r is only half of the test (r½). To determine the reliability of our test as a whole we use Spearman-Brown formula. Sample computation is shown below.

Analysis: the result above shows that 0.98 is closer to +1 hence the test is of excellent reliability. As a guide to the to interpret the coefficients of stability that are between 1 and 0, refer it below:

• 0.9 and greater: excellent reliability 
• Between 0.9 and 0.8: good reliability 
• Between 0.8 and 0.7: acceptable reliability 
• Between 0.7 and 0.6: questionable reliability
• Between 0.6 and 0.5: poor reliability

Writing Anecdotal Record

An anecdotal record is an observational strategy utilized every now and again in classroom settings in which the onlooker condenses a solitary formative episode after the occasion has happened. Composed from memory, the anecdotal record archives the student's development and patterns. Its writing a like short stories; it should have a beginning, a middle and an end.

Teachers generally make use of anecdotal records because its easy to utilize and quick to write. It permit teachers to record qualitative data such as child's particular behavior or the discussion between two learners. These information can help teachers design classroom activities and interventions. This will also supplement classroom assessment of teachers such as oral questioning and performance test.

Anecdotal records allow teachers to:

• assess development in physical, social, cognitive, and emotional areas;
• learn more about  the child as an individual;
• identify interests for the group or the individual;
• remember information;
• show progress and share information;
• assess the curriculum;
• gather data to drive various decisions
• provide information regarding a child's development over a period of time
• provide ongoing records about individual instructional needs
• capture observations of significant behaviors that might otherwise be lost
• provide ongoing documentation of learning that may be shared with students, parents and teachers

Some Guidelines for Writing Anecdotal Records:

1. Start with a statement, setting, date, time of day, name, and age of child
2. Describe the child's behavior NOT what you think of the behaviors
3. Use details of the child's behavior such as actions or comments
4. Write down the exact words used in the conversation
5. Being positive and objective, and using descriptive language are also important things to keep in mind when writing your anecdotal records. 

Different Areas that can be observed by Teachers

1. Physical Domain
• refers to behaviours involving the body
• can be grouped into gross-motor (large muscles) and fine-motor (small muscles) behaviours, as well as sensory skills (hearing, seeing, tasting, smelling, touching)
• Some examples of skills from the physical domain:
• runs with control over speed 
• draws straight lines 
• catches a large ball
2. Language Domain:
• refers to behaviours involving communicating (listening, talking, non-verbal communication)
• Some examples of skills from the language domain: 
• names familiar objects 
• enjoys looking at books
• uses two- word sentences
3. Aesthetic Domain
• refers to skills and behaviours used to perceive, respond to and evaluate beauty, both in the natural world and man-made objects
• Some examples of skills from the aesthetic domain: 
• explores new textures 
• shows a colour preference
• shares ideas about what is beautiful
4. Cognitive domain
• refers to skills used to know, learn and understand
• Some examples of skills from the cognitive domain: 
• follows instructions 
• names geometric shapes 
• sorts objects
5. Emotional Domain
• refers to the "inner life" of the child
• important elements of the Emotional domain are self-concept, self-esteem and autonomy
• Some examples of skills from the emotional domain:
• seeks out new experiences and materials
• separates easily from parent in the morning
• seeks out their transitional object (blanket, pacifier...) when frightened or upset
6. Social Domain
• refers to behaviours and skills used to establish and maintain relationships with others
• Some examples of skills from the social domain:
• initiates play with other children
• shares toys with peers
• has one or two preferred playmates

Parts of a Written Anecdotal Record

Sample Anecdotal Record

References:

http://toddlers.ccdmd.qc.ca/observing/anecdotal_records

http://www.learnalberta.ca/content/mewa/html/assessment/anecdotalnotes.html