Item Analysis

What is an Item Analysis?
Item analysis examines student responses to individual test items or test questions in order to determine the quality of those items. It is valuable in improving test items which can be used again in later tests. It can be also used to eliminate misleading test items in a single test administration. Moreover, it will enhance instructors' skills in test construction, and identifying specific areas that need greater emphasis.

Classical Item Analysis Statistics
The following are item analysis statistics that you can use:

• reliability - determines the consistency of the test (test level statistics)
• difficulty - can be inferred from the number of students who answered the item correctly (item level statistics)
• discrimination - can be inferred from the number of respondents in the lower group who got the correct answer on the test item (item level statistics)

How to Item Analyze?
1. Score the test papers to obtain the total scores of the students
2. Arrange test scores of the students from highest to lowest
3. Split the test papers into a high group and low group

• For a class of ≤ 30 do a 50-50 split, For example, if your N=20, the upper 10 represent your upper group (nU) and the lower 10 represent your lower group (nL)
• For a large group, you may take the upper (nU) 25-27% and lower (nL) 25-27% (Nunnally, 1972; Wiersma & Jurs, 1990). For example: If there are 100 pupils taking the test, the nU (27%) = 27 pupils and nL (27%) = 27 pupils

4. Compute the index of difficulty by calculating the p-value (p) which is the proportion of students answering the item correctly.

p=R/T

where: p is the p-value; R is the total number of students answering the item right, and T is the total number of students answering the item

or you can use this to calculate:

D= [(U/nU+L/nL))]/2

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

5. Compute the discrimination index. Obtain first the p-value for the upper and lower group, then get the difference between the p-values. 

p=(Rᵤ/Tᵤ)-(Rₗ/Tₗ)

where: p is the p-value; Rᵤ is the total number of students answering the item right in the upper group; Tᵤ is the total number of students answering the item in the upper group; Rₗ is the total number of students answering the item right in the lower group; Tₗ is the total number of students answering the item in the lower group

or you can use this to calculate:

ID= (U/nU)-(L/nL)

where: U is the number of students who responded to the item correctly in the upper group; nU is the total number of students in the upper group; L is the number of students who responded to the item correctly in the lower group; nL is the total number of students in the lower group

Example: (using 25%)

The correct answer to a test item in a Science test is “b”. There were forty pupils in the class. The upper 25% consists of the top 10 pupils and the lower 25% consists of the lowest 10 pupils. Eight pupils from the upper group got the item correctly, and only 3 in the lower group got the correct answer. What is the index of difficulty and discrimination index of this test item? What action can be taken based on the results of D & ID? 

Solution:

D=[(U/nU+L/nL)]/2

D=[(8/10+3/10)]/2

D=(.8+.3)/2

D=1.1/2

D=0.55 (moderately difficult item)

ID= (U/nU)-(L/nL)
ID=(8/10)-(3/10)
ID=.8-.3
ID=0.50 (discriminating item)

5. Interpret the result of D and ID. To do this, you may use the difficulty & discrimination index table by Hopkins & Antes (1990) or the table below:

Using the table above, D=.55 means that the item is of moderate difficulty and ID=.50 means it is discriminating item. Thus, the action (moderately difficult🠖discriminating) is "include" the item. 

References:

1. Santos, R. G. (2007). Assessment of Learning 1. Quezon City: Lorimar 
2. Padua, R.N. and Santos, R. G. (1997). Educational Evaluation and Measurement: Theory, Practice, and Application. KATHA Publishing: QC.

Internal Consistency Reliability: Split Half Method

Internal Consistency Reliability is a subdivision of the test into two or more equivalent parts.  In internal consistency reliability, we judge the reliability of the instrument by estimating how well the items that reflect the same construct yield similar results. 

One method we can use is split-half. In split-half reliability, we randomly divide all items that purport to measure the same construct into two sets. How? 

First, divide the test into half usually using the odd-even technique. As shown in the illustration below, the items were divided into odd-even numbered items, where odd-numbered items (1, 3, 5) forms the "x" group while even-numbered items (2, 4, 6) forms the "y" group.

Second, find the correlation of scores using Pearson r formula. To do this, make a table as shown in the example below.
Example: Four pupils took a 50-item test. Below are the results

The table above shows that odd-numbered items are denoted by (x) while even-numbered items are denoted by (y). The rows in the two columns show the scores of the four (4) pupils who took the 50-item test. For example, pupil 1 scored a total of 30 out of 50 in the test. When divided into odd-even numbered items, it shows odd=10 and even=20. Likewise, pupil 2 scored only 13 out of 50 in the test where odd-numbered items=5 and even-numbered items=8.

Moreover, the first two columns are given based on the result of the test while the last three columns need to be solved. For the third and fourth columns, all you have to do is to square the x and y variables. The last column is the product of x and y. 

Finally, the last row provides the sum of all the columns. Once you have all this data you can now substitute the Pearson r equation. Just follow the example below.

Third, adjust & re-evaluate correlation using Spearman-Brown formula

The purpose of re-evaluating the correlation is to determine the reliability of the test as a whole. Since we divide the test into half, the result of the correlation in step 2 using Pearson r is only half of the test (r½). To determine the reliability of our test as a whole we use Spearman-Brown formula. Sample computation is shown below.

Analysis: the result above shows that 0.98 is closer to +1 hence the test is of excellent reliability. As a guide to the to interpret the coefficients of stability that are between 1 and 0, refer it below:

• 0.9 and greater: excellent reliability 
• Between 0.9 and 0.8: good reliability 
• Between 0.8 and 0.7: acceptable reliability 
• Between 0.7 and 0.6: questionable reliability
• Between 0.6 and 0.5: poor reliability

Writing Anecdotal Record

An anecdotal record is an observational strategy utilized every now and again in classroom settings in which the onlooker condenses a solitary formative episode after the occasion has happened. Composed from memory, the anecdotal record archives the student's development and patterns. Its writing a like short stories; it should have a beginning, a middle and an end.

Teachers generally make use of anecdotal records because its easy to utilize and quick to write. It permit teachers to record qualitative data such as child's particular behavior or the discussion between two learners. These information can help teachers design classroom activities and interventions. This will also supplement classroom assessment of teachers such as oral questioning and performance test.

Anecdotal records allow teachers to:

• assess development in physical, social, cognitive, and emotional areas;
• learn more about  the child as an individual;
• identify interests for the group or the individual;
• remember information;
• show progress and share information;
• assess the curriculum;
• gather data to drive various decisions
• provide information regarding a child's development over a period of time
• provide ongoing records about individual instructional needs
• capture observations of significant behaviors that might otherwise be lost
• provide ongoing documentation of learning that may be shared with students, parents and teachers

Some Guidelines for Writing Anecdotal Records:

1. Start with a statement, setting, date, time of day, name, and age of child
2. Describe the child's behavior NOT what you think of the behaviors
3. Use details of the child's behavior such as actions or comments
4. Write down the exact words used in the conversation
5. Being positive and objective, and using descriptive language are also important things to keep in mind when writing your anecdotal records. 

Different Areas that can be observed by Teachers

1. Physical Domain
• refers to behaviours involving the body
• can be grouped into gross-motor (large muscles) and fine-motor (small muscles) behaviours, as well as sensory skills (hearing, seeing, tasting, smelling, touching)
• Some examples of skills from the physical domain:
• runs with control over speed 
• draws straight lines 
• catches a large ball
2. Language Domain:
• refers to behaviours involving communicating (listening, talking, non-verbal communication)
• Some examples of skills from the language domain: 
• names familiar objects 
• enjoys looking at books
• uses two- word sentences
3. Aesthetic Domain
• refers to skills and behaviours used to perceive, respond to and evaluate beauty, both in the natural world and man-made objects
• Some examples of skills from the aesthetic domain: 
• explores new textures 
• shows a colour preference
• shares ideas about what is beautiful
4. Cognitive domain
• refers to skills used to know, learn and understand
• Some examples of skills from the cognitive domain: 
• follows instructions 
• names geometric shapes 
• sorts objects
5. Emotional Domain
• refers to the "inner life" of the child
• important elements of the Emotional domain are self-concept, self-esteem and autonomy
• Some examples of skills from the emotional domain:
• seeks out new experiences and materials
• separates easily from parent in the morning
• seeks out their transitional object (blanket, pacifier...) when frightened or upset
6. Social Domain
• refers to behaviours and skills used to establish and maintain relationships with others
• Some examples of skills from the social domain:
• initiates play with other children
• shares toys with peers
• has one or two preferred playmates

Parts of a Written Anecdotal Record

Sample Anecdotal Record

References:

http://toddlers.ccdmd.qc.ca/observing/anecdotal_records

http://www.learnalberta.ca/content/mewa/html/assessment/anecdotalnotes.html

Developing Discussion Techniques: Panel Forum

Discussion techniques help create a conducive atmosphere
A panel forum usually involves four to six students who discuss a topic in front of the class followed by comments or reactions from the class. The steps to carry this out involves the following:

1. Identify the particular subject matter through assigning it in advance or brainstorming in class
2. Divide the class into groups or groupings. It is an ideal scenario if small groups are created to allow all members to share.
3. Each group / grouping must have a leader who will serve as the moderator and rapporteur. Such task must be assigned to a capable and active student who can communicate well both in oral and written English or Filipino. The spirit of volunteerism must be encouraged in order to create an atmosphere of trust and respect.
4. Once a leader in a group is defined, there should be no time wasted by seeing to it that the discussion will start on time.
5. It is a must that the subject matter, topic or problem be evaluated in terms of its data or statistics from reliable sources.
6. Concluding activities must not be overlooked as this brings finality to what occured in the discussions.

The panel forum is commonly practiced in television shows where the opinions or viewpoints of the participants are magnified through the use of television cameras and microphones. A panel forum moderator makes sure that all the differing viewpoints are expressed in such a manner that both sides of a subject, especially controversial ones, are given enough exposure. The moderator makes sure that nobody dominates the conversation. Watch the videos on panel forum, see how the moderator manages the discussions and the experts/specialists tackle the questions or the topic.

Panel Forum: Bullying

Panel Forum: Future of Sales 

Course Syllabus for Assessment of Learning 1

A. COURSE INFORMATION

Course Number	ED 127 – Assessment of Student Learning 1
Course Credit	3 units
Course Component, Hours/Week	3 hours/week
Course Description	This course focuses on the development and utilization of assessment tools to improve the teaching-learning process. Emphasis is given on the use of Objective Testing for measuring knowledge and thinking skills of students in the Basic Education
Course Goals	(1) Show understanding of the basic concepts and principles of high quality assessment; (2) Match the different learning targets & their appropriate assessment methods; (3) Identify the role of assessment in making instructional decision; (4) Construct classroom assessment tools in measuring knowledge and thinking skills; (5) Derive information from test results; (6) Apply statistical procedures in standardizing test
Pre-requisite	Math 31		Co-requisite	FS 104
Reference/s	(1)Del Socorro, FR., Omas-as, RL., & Galela, RS. (2011). Assessment of Student Learning 1 & 2. Great Books Publishing. (2) Santos, R. d. (2007). Assessment of Learning 1. Quezon City: Lorimar; (3) Kubiszyn, T., & Borich, G. (2000). Educational Testing and Measurement: Classroom Application and Practice. New York: John Wiley & Sons; (4) Wegener, D. P. (2009). Test Construction. Available online at: http://www.delweg.com/dpwessay/tests.htm
Other Supplemental Materials	PELC/PSSLC K-12 Curriculum Guide
Course Requirements	individual task (15%); group task (15%); project (20%); quizzes (15%); term exam (25%); attendance (10%)
Grading System	Class Participation (75%) 1.       Attendance 2.       Project 3.       Quizzes/Seatwork/Exercises 4.       Individual & Group Task (online/offline/oral recitation) Term Exam (25%)
Teacher	Name: Room: Schedule:

 B. LEARNING PLAN

Desired Learning Outcome	Learning Content	Teaching and Learning Activities	Instructional Materials & References	Assessment Tasks & Tools	Evidence of Outcomes	Time Allotment
1.       Explain the VMGO of the Institute & of the College 2.       Discuss course coverage, grading system and their importance	1.        VMGO of MSU-IIt & of the College of Education 2.        Course Coverage and Grading System	·   Reflective Thinking ·   Group Discussion	-LCD Projector -PPT Presentation Student Manual	·   reflective journal ·   oral questioning *rubric	All students achieve at least _____%	3 hours
3.        Describe educational measurement, assessment, and evaluation 4.        Identify types of educational measurement, and roles of assessment 5.        Distinguish indicators, variables, & factors	3.        Basic Concepts: Educational Measurement; Assessment; Evaluation of Learning & Programs 4.        Types of Measurement §  indicators, variables &  factors 5.        Various roles of assessment	·   Class Discussion ·   Buzz Group Discussion ·   Situation Analysis	-LCD projector & laptop -PowerPoint presentation / flipchart ASL1 pp. 7-13	·   K-W-L-H ·   Oral Questioning ·   Oral Presentation *rubric *MOLE quiz module	All students achieve at least _____%	3 hours
6.        Compare behavioral  and non-behavioral objectives 7.        Explain clarity of learning targets /objectives 8.        Write a specific and clear learning targets / objectives 9.        Discuss the learning domains 10.     Classify cognitive targets / objectives according to Bloom’s taxonomy	6.        Clarity of Learning Targets - cognitive targets - skills, competencies & abilities targets 7.        Learning Domains 8.        Bloom’s Taxonomy of cognitive objectives	·   Visual List: In groups, students list down the given objectives under proper column labelled behavioural │ non-behavioral objectives ·   Collaborative Learning-writing SMART learning objectives	-LCD projector & laptop -PowerPoint presentation / flipchart -Hand-out on learning domains -Chart on Bloom’s taxonomy of cognitive objectives ASL1 pp. 20-23	·   Group presentation ·   Performance test (writing learning targets) *rubric *performance checklist *MOLE quiz module	·   All students achieve at least _____%	6 hours
11.     Discuss various assessment methods 12.     Determine appropriate assessment methods relative to the learning targets	9.        Appropriateness of Assessment Methods - written response    instruments - product rating scales - performance test - oral questioning - observation and self-reports	·   Lecture-Discussion: ·   Panel Discussion: Student groups are assigned to write an objective, determine appropriate assessment method  and asked to make a short presentation	-LCD projector & laptop -PowerPoint presentation -Hand-out on assessment methods -Video on importance of observation & self-reports ASL1 pp. 24-26	·   Oral Questioning ·   Group Presentation ·   Prelim Exam *rubric *MOLE quiz module / on site exam	All students achieve at least _____%	3 hours
Desired Learning Outcome	Learning Content	Teaching and Learning Activities	Instructional Materials & References	Assessment Tasks & Tools	Evidence of Outcomes	Time Allotment
13.     Describe what is validity & reliability of test 14.     Discuss the following: ·  content validity ·  criterion related  validity ·  construct validity ·  test-retest reliability ·  parallel forms reliability ·  internal consistency reliability using split half method ·  fairness of assessment ·  practicality & efficiency of assessment ·  ethics of assessment 15.     Establish validity & reliability of a given test	10.     Properties of Assessment Methods ·         validity of test ·         reliability of test ·         fairness of assessment ·         practicality & efficiency of assessment ·         ethics in assessment	·   Group Research on assigned topics ·   Asynchronous Learning: Group members participate in online forum on assigned topics ·   Collaborative Learning: Groups members participate in validating a given test	-LCD projector & laptop -PowerPoint presentation -Hand-out on properties of assessment -Flipchart ASL1 pp. 26-31	·   Group Presentation ·   Peer Assessment ·   Performance Test *Rubric *performance checklist *MOLE forum activity module	All students achieve at least _____%	7 hours
16.     Identify steps in planning for a test 17.     Discuss on how to construct a table of specifications (TOS) 18.     Construct Table of Specifications	11.     Planning a Test ·         steps in planning for a test ·         table of specifications	·   Lecture-Discussion ·   Student-Teacher Interaction ·   Workshop: Group members participates in planning and constructing their respective TOS	-LCD projector & laptop -PowerPoint presentation -Chart on table of specification ASL1 p. 37	·   Oral Questioning ·   Group Presentation ·   Performance Test *rubric *performance checklist	All students achieve at least _____%	3 hours
19.     Identify and describe types of objective & free-response tests 20.     Discuss guidelines in constructing objective & free response tests 21.     Construct objective & free response type of tests 22.     Discuss guidelines in creating scoring guide for free response test 23.     Create scoring guide for free responses test	12.     Types of Tests ·         true-false type test ·         matching type test ·         supply type test ·         multiple choice test ·         short answer test ·         essay test	·   Group Research on assigned topics ·   Class Discussion ·   Workshop1: Group members participates in planning and constructing objective and free response tests ·   Workshop2: Group members actively participates in creating scoring guide for free-response test following the guidelines	-LCD projector & laptop -PowerPoint presentation -Hand-outs on types of test -PELC/PSSLC or Curriculum Guide -Chart on rubrics ASL1 pp. 36, 39-49	·   Group Presentation ·   Oral Questioning ·   Product Outputs ·   MC test ·   Midterm Exam *rubric *product rating scale *MOLE quiz module / on site exam	All students achieve at least _____%	9 hours
Desired Learning Outcome	Learning Content	Teaching and Learning Activities	Instructional Materials & References	Assessment Tasks & Tools	Evidence of Outcomes	Time Allotment
24.     Define what is item analysis 25.     Describe steps in item analysis 26.     Discuss index of difficulty and discrimination 27.     Analyse items in a test constructed	13.     Item Analysis ·         index of difficulty ·         index of discrimination	·   Lecture-Discussion ·   Student-Teacher Interaction ·   Collaborative Learning: Group members participates in Item Analysis of the test constructed	-LCD projector & laptop -PowerPoint presentation -Hand-outs on item analysis ASL1 pp. 54-65	·   Oral Questioning ·   Group Outputs *rubric *checklist	All students achieve at least _____%	5 hours
28.     Discuss the following measures of central position 29.     Compute the measures of central position in a given data	14.     Measures of Central Position ·         Mean ·         Median ·         Mode	·   Lecture-Discussion ·   Student-Teacher Interaction ·   Think/Pair/Share	-LCD projector & laptop -PowerPoint presentation -Calculator ASL1 pp. 66-70	·   Oral Questioning ·   Numerical Exercises *Quiz *MOLE quiz module	All students achieve at least _____%	4.5 hours
30.     Discuss the measures of variability 31.     Determine the homogeneity or heterogeneity of a given data	15.     Measures of Variability ·         fractiles ·         quartile deviation ·         mean absolute deviation ·         standard deviation & variance	·   Lecture-Discussion ·   Student-Teacher Interaction ·   Group Dynamics	-LCD projector & laptop -PowerPoint presentation -Hand-outs -Calculator ASL1 pp. 66-75	·   Oral Questioning ·   Numerical Exercises *Quiz *MOLE quiz module	All students achieve at least _____%	4.5 hours
32.     Identify & describe various grading system 33.     Use grading system in converting raw score values into standard grading formats	16.     Grading System ·         norm-referenced grading ·         criterion-referenced grading system ·         alternative grading system ·         cumulative & averaging grading system	·   Student-Teacher interaction ·   Collaborative Learning - converting raw scores into standard grading formats	-LCD projector & laptop -PowerPoint presentation -Class record ASL1 pp. 77-88	·   Oral Questioning ·   Group Outputs ·   Final Exam *rubric *MOLE quiz module / on site exam	All students achieve at least _____%	6 hours