Constructing Objective Tests: MC, T-F, Fill-in the blanks, Matching Type Test, Jumbled Letter and Cloze Test

Objective Tests are either structured response tests or restricted-response tests. Structured-response tests are those for which the responses are indicated in the tests item itself and the test taker needs to choose the answer from the possible responses. Restricted-response test is also an objective type for which no ready-made responses are given but each item has only one answer.

Multiple Choice Test

This test consists of two parts: the stem and the option/alternatives (consists of answer or key and distractors). In constructing this kind of test we should remember the following:
1. The stem must express a complete thought.

Example 1: The poem “The Raven”
a) was written by Edgar Alan Poe
b) was written by Elizabeth Browning
c) Was written by Omar Khayyam
d) Was written by Jose Garcia Villa

Example 2 The poem “The Raven” was written by
a) Edgar Alan Poe
b) Elizabeth Browning
c) Omar Khayyam
d) Jose Garcia Villa

The second example is better than the first since the stem contains a complete thought.

2. Keep options short while putting most of the concepts in the stem
3. The stem must contain only the information necessary to make the problem clear. Do not provide unnecessary details or worse provide a clue to the answer.

Example 1: The revolution in Phil. to oust President Marcos took place after a snap election in 1986. It happened at the dawn of Feb. 23. When did the revolution take place?
a) Before Valentine’s day
b) After Valentine’s day
c) On Valentine’s day
Example 2: When did the People’s Power Revolution in Phil. take place?
a) Feb., 1985
b) Feb., 1986
c) Feb., 1987

The first example does not measure knowledge of Philippine History instead, it focuses on knowledge of Valentine's day. Moreover, the stem provided the clue to the answer Feb 23 is after Feb 14. The second example provides a better choice than the previous one.

4. Avoid negative statements or double-negative statements in the stem. This may confuse the test taker.
Example 1: It is not untrue that Magellan discovered the Phil
Example 2: It is true that Magellan discovered the Phil

The second example is better than the previous since example 1 contains a double-negative statement.
5. Make sure that there is only one correct answer to the question. Students panic if they find several correct answers to the question.

Example: When did the people’s Power Revolution take in the Phil.?
a) Feb. 23, 1986
b) After the snap election
c) March 1, 1986
d) After Valentine’s day in 1986

In this example, there are three possible answers to the questions: A, B, and D

6. Make options equally attractive. This means that the correct answer should be made equally plausible as the "distractor" otherwise the answer will stand out like a sore thumb.

Example: The author of “The Raven” is
a) Jose Garcia Villa
b) Edgar Alan Poe
c) Genoveva Matute
d) Francisco Balagtas

In the example, all except letter (b) are Filipino authors. Since the poem is very foreign-sounding to the students the author must be a foreigner.

7. Use the option "none of these" or "none of the above" only when there is only one correct answer.
8. Ensure that items do not dwell too much on "knowledge" or rote learning. MC items when properly constructed can elicit high order responses. The example below shows that the item measures both comprehension and analysis.

Example: The volume of a sphere is given by v=4/3 (pi) r^3 where r=radius of the sphere? The volume will be:
a) Multiplied by a factor of 2
b) Multiplied by a factor of 3
c) Multiplied by a factor of 4
d) Multiplied by a factor of 8

9. As much as possible avoid using "all of the above" as an option

Varieties:

1. Single Correct Answer
2. Best Answer
3. Negative Stem
4. Multiple Response
5. Combined Resources
6. Stimulus-Material-Stem-Alternatives

True or False Test

The T-F test measures only knowledge facts. It is presented in a form simple declarative statement. to which the test taker responds indicating whether the statement is true or false. The following are some types of T-F test.

• simple T-F
• modified T-F
• T-F with correction

To construct such a test there are few things to remember:

1. Each item should refer to only one concept. This means that students must know which concept is being referred to.

Example: Columbus discovered Phil in 1721 (TF)

The example is ambiguous because the taker does not know if he will answer T-F to Columbus discovering the Phil or the year 1721

2. TF items must not include an option statement. When students were asked to respond TF to an opinion they get confused since there is no objective basis for determining whether the opinion is true or false.

Example 1: The Phil Constitution is better than the Malaysian Constitution
Example 2: Part of the constitution of Phil and Malaysia are the same

The second example is better than the previous since it eliminates an option statement.

3. Avoid specific quantifiers. Existential quantifiers such as some, usually, often, generally, are often TRUE while universal quantifiers such as always, never, all, none are often FALSE. A test wise student will detect these patterns and will respond correctly even if they do not know the answer.

4. Avoid a long and complex sentence. Long sentences do not measure knowledge of facts but they do measure the student’s reading comprehension.

5. Avoid using a pattern for the occurrence of TF statements

Fill in the Blank Test
This is a test question where students are given a statement with a blank and they are required to fill it in with the most appropriate answer possible. It requires mostly the remembering level of Bloom’s Taxonomy and if well designed a question can test higher-order thinking. This test consists of a stem and a blank where the students would write the correct answer.

example¹: The study of life and living organism is called _______.

example²:  Write the missing letters to complete the number word. [16 - s_ _ _ _ _ n]

example³: What types of noun are the given words? 

person

place

animal

thing

1. cat - _______

2. book - ______

Advantages:

• Fill in the blanks questions allow for a wide sampling of content.
• It also minimizes the chance of students guessing the answer. Fill in blank requires students to think about the correct plausible answer, as opposed to choosing from multiple possible answers.

Disadvantages:

• Fill in the blank questions are usually recall-knowledge based questions.
• Marking questions can become time-consuming as there could be multiple unique answers which are all potentially correct.
• It is difficult to create a fill in the blank question that can only result in one answer unless there are relevant clues.
• Providing an answer key for students to choose from can help eliminate the multiple answer possibility, but limits the thinking process of answering the question.

(source: https://sites.google.com/site/creatingtestquestions/home/4-fill-in-the-blank)

Matching Type Test
The match type items may be considered as modified multiple-choice type items. It is an item that provides a defined term and requires a test taker to match identifying characteristics to the correct term. It has two columns with each concept in one column being matched with a concept in the other column. Items for which match is right are called PREMISES and the column from which to choose the match is the RESPONSE column. To construct such test the following should be noted:

1. Be sure that the materials in the test are homogenous. This means that we should be testing a concept at a time for each item…in matching case we should specify the concept pair we are trying to test…for instance province-capital pair in the Phil.

Example: For each province in Column A, find the corresponding capital in column B. Write the letter corresponding to the capital on the blank provided for in column A.

Column A                        Column B
___1. Misamis Occ           a. Butuan City
___2. Agusan del Norte    b. Ozamis City
                                          c. Cagayan de Oro City

2. Use an imperfect match type as much as possible. A perfect match will encourage guessing.
3. Indicate the basis for matching column A and column B. Use directions effectively to explain clearly the basis for columns A and B.
4. Place all of the items for one matching exercise on the same page.

When to use?
Can be used when the content has parallel concepts such as:

1. terms and definitions
2. objects/pictures and labels
3. symbols and proper names
4. causes and effects
5. scenarios and responses
6. principles and scenarios to which they apply

Jumbled Letter Test
A jumble letter test is a type of puzzle where a sequence of letters in a word or group of words are changed. For example, you re-arrange the order of letters in the word "river" to "evrir". It can be also played by scrambling the letters to form an anagram. For example, the word "cinema" is re-arrange to form "iceman". The goal with this type of test is to figure out the correct order of the word/s to form a correct word or sentence. This type of test has no specific rules. Below are some examples of this test or exercise.

1. simple jumbled spelling test
• Disentangle these sprotsman
• treslerw (wrestler)
• remwims (swimmer)
• urnenr (runner)
2. terminological jumbled letters
• Unscramble the following word based on the hints:
• oatnby (study of plants)
• ihyttglcohoy (study of fishes)
3. re-arranging jumbled letters to get desired paragraph, sentence, word or thought
• i cdnuolt blveiee taht I cluod aulaclty uesdnatnrd waht I was rdanieg. The phaonmneal pweor of the hmuan mnid, aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it dseno’t mtaetr in waht oerdr the ltteres in a wrod are, the olny iproamtnt tihng is taht the frsit and lsat ltteer be in the rghit pclae.. The rset can be a taotl mses and you can sitll raed it whotuit a pboerlm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. Azanmig huh? yaeh and I awlyas tghuhot slpeling was ipmorantt!.

Cloze Test

A Cloze Test (also called the "cloze deletion test") is an exercise, test, or assessment consisting of a portion of text with certain words removed (cloze text), where the teacher asks the participant to restore the missing words. Cloze tests require students to understand context and vocabulary to identify the correct words that belong in the deleted passages of a text.

EXAMPLE:

A language teacher may give the following passage to students:

"Today, I went to the ________ and bought some bread and peanut butter. I knew it was going to rain, but I forgot to take my ________, and got wet on the way."

The teacher instructs the students to fill in the blanks with words that they think best fits the passage. Both context in language and content terms are essential in most cloze tests. The first blank is preceded by "the"; therefore, a noun, an adjective or an adverb must follow. However, conjunction follows the blank; the sentence would not be grammatically correct if anything other than a noun was in the blank. The words "bread and peanut butter" is important for deciding which noun to put in the blank; "supermarket" is a possible answer; depending on the student, however, the first blank could either be store, supermarket, shop or market, while umbrella or raincoat fit the second.

The Development of the Cloze Test

Research indicates that teachers at many elementary schools require their students to read books and materials that they often struggle to read. This condition is largely based on the graded system which assumes that all children learn all things at virtually the same time. It seems imperative that teachers choose materials which match the students' reading skills.

To accomplish this, the first task is to determine the appropriateness of reading materials for various students. To some extent, the standardized achievement tests offered at least once a school year in most school systems, provide such information. However, the results of such tests do not provide a reliable index of reading success in various materials.

The reasons for this are:

1) Achievement tests are based on limited samples; they cannot predict achievement accurately in specific materials that draw on varied concepts, sentence patterns, etc.

2) Achievement tests are most reliable in the middle ranges of achievement. They often mislead in measuring the achievement of those in the lower reading ranges.

Because standardized tests cannot accurately determine the suitability of given reading materials, many reading authorities suggest informal tests of the involved materials. The best test of reading skill relies on the student's ability or inability to read the given material.

Thus, if a sixth-grade teacher wishes to find out which students can read and comprehend the sixth-grade geography text, the teacher must:

1. Direct each student to read a specified portion of the text.
2. Direct the student to demonstrate some degree of understanding. A student can do this by answering questions about the selection.

This method of testing materials is generally called "informal reading inventory testing." In most instances, the label is equated with the task of finding pupils' reading levels by asking them to read a series of increasingly difficult selections (followed by comprehension questions).

Students in the earlier stages of reading development read the various materials both orally and silently, while higher-level students read silently before answering the questions.

Although potentially valuable, "informal reading inventory testing" involves many qualitative decisions on the part of the teacher, such as:

1) Oral Reading

• Mutes are oral reading errors?
• What is the maximum number of oral reading errors that can be permitted?
• How fluent should oral reading be?
• How do you determine fluency?

2) Silent Reading

• What is a reasonable amount of time to read the given selection?

3) Comprehension

• What are the most important elements that the student should remember about the selection?
• To what extent are the questions relevant to the main elements of the selection?

The quality of judgments in the above depends upon very sophisticated judgments. In fact, the judgments can be so sophisticated that reading experts suggest that teachers may make completely inappropriate judgments if they use the prevailing error marking systems.

At this point, the question many teachers ask is, "If teachers cannot depend upon achievement tests or their own observations to determine the suitability of reading materials for different children, what, then can they use?'"

We have two very different ways. Several diagnostic reading test authors have developed tests that can more accurately predict the proper instructional level of texts and others have presented data to indicate that their special instruments will predict more accurately than achievement tests. Another way has been seen in the "cloze technique" procedure as developed by John Bormuth (1967).

In the "Cloze Test Procedure," the teacher instructs students to restore omitted words (usually every fifth word) in a reading passage. Based on reviewing students' restored words from the text passages, the teacher can determine a more accurate level of comprehension.

Because of the the Botel Readability Formula and Spache Readability Formula (as well as other formulas) suffer from the same limitations as achievement tests, it appears that their usefulness to determine the appropriateness of reading material is limited.

The Cloze Test is different. Devoid of such restraints and geared to the exact material, the Cloze Test Procedure adds more value to determine the readability of any selected text for any student.

Wilson L. Taylor introduced the term "cloze procedure" in 1953 and thoroughly researched the value of closure tasks as predictors of reading comprehension. Basic to the procedure is the idea of closure wherein the reader must use the surrounding context to restore omitted words. Comprehension of the total unit and its available parts (including the emerging cloze write-ins) is essential to the task.

To use the Cloze Test Procedure to score material, follow this protocol:

Administration
1. Omit every 5th word, replacing it with a blank space for the student to write in the answer.
2. Instruct students to write only one word in each blank and to try to fill in every blank.
3. Guessing is encouraged.
4. Advise students that you will not count misspellings as errors.

Scoring
1. In most instances, the exact word must be restored.
2. Misspellings are counted as correct when the response is deemed correct in a meaningful sense.

Validating the effectiveness of the Cloze Test as a measure of readability and comprehension is interesting because of: (1) the ways in which reading comprehension is scored; and (2) the almost universal finding of high correlations between cloze and other prediction instruments.

Initially Taylor (1953) compared cloze score rankings of passages of varying difficulty with readability rankings of the same passages by two common readability formulas, Dale-Chall and Flesch formulas. The passages were similarly rank-ordered by each technique. The Cloze Test scored the readability of very difficult text passages more accurately than the Dale-Chall and Flesch formulas.

Some rules of thumb exist for how long it takes most students to answer various types of questions:

• A true-false test item takes 15 seconds to answer unless the student is asked to provide the correct answer for false questions. Then the time increases to 30-45 seconds.
• A seven-item matching exercise takes 60-90 seconds.
• A four response multiple-choice test item that asks for an answer regarding a term, fact, definition, rule or principle (knowledge level item) takes 30 seconds. The same type of test item that is at the application level may take 60 seconds.
• Any test item format that requires solving a problem, analyzing, synthesizing information or evaluating examples adds 30-60 seconds to a question.
• Short-answer test items take 30-45 seconds.
• An essay test takes 60 seconds for each point to be compared and contrasted.

References:

1. Santos, R.G. (2007) Assessment of Learning 1. Lorimar:QC
2. Padua, R.N. & Santos, R.G. (1997). Educational Evaluation & Measurement: theory, practice, and application. Katha Publishing: QC.

Source: ReadabilityFormulas.com