College Teaching, Fall 1997 v45 n4 p150(3)
Essays - well worth the effort. Craig W. Steele.
Abstract: More college teachers should make use of essay examinations in their courses. Well-constructed essay questions test higher-level knowledge than most objective questions. They also furnish students with valuable writing practice and provide the opportunity for mutual feedback between teacher and student. Some disadvantages of essay exams include the amount of time they take to grade and their inherent subjectivity. Teachers can increase their grading objectivity by covering students' names, deciding in advance the key points each essay should cover, and stopping the grading when they begin to feel tired.
Full Text: COPYRIGHT 1997 Heldref Publications
Why do questions requiring essay responses make students so apprehensive? Because they are used to taking multiple-choice tests. By the time they enter college, students have had up to twelve years training in this type of examination; it is what they expect in most classes. Multiple-choice tests are easier to take (usually involving only simple recognition and recall), permit a certain amount of guessing, and require basic study skills.
But multiple-choice exams, in my opinion, are the least appropriate evaluation tools for aiding student learning, at any grade level. They do little to help teachers "teach for understanding" (Perkins 1993). I agree with the position of the Writing across the Curriculum (WAC) Association of the Pennsylvania State System of Higher Education, as expressed in the May 1991 issue of Writing and Learning, that "writing is central to all disciplines" because an "active language element is crucial for any significant learning."
Why then, according to WAC, do "most college classes not only neglect the language-based elements of learning, but actively avoid them?" Perhaps the reason is that teachers, as well as students, do not appreciate the advantages of the essay. Also, according to Everett (1994), many teachers avoid writing assignments because of the perceived difficulty and subjectivity in grading them.
Advantages of the Essay
Essays can test higher-level cognitive skills. Objective exams can also test higher-level skills, but on a more limited basis. For any test, the content and style of the questions should be determined by the course objectives. When objectives require students to apply knowledge, analyze data or situations, synthesize, or evaluate, the essay can be an effective test. Because essays address higher-level cognitive processes, you can individually direct students in their thought processes. However, when objectives ask students to describe, to list, or to recognize, the essay exam is probably not useful.
Students study more efficiently in preparing for an essay test. Research has shown that students generally spend less time on rote memorization of material when they study for an essay exam rather than a multiple-choice exam (e.g., Mayer 1975; Shavelson and Stern 1981). Instead, students tend to generalize and conceptualize the material, using facts as support material rather than as ends unto themselves. Students are al so more apt to retain concepts and generalizations than isolated facts and details.
Essays allow students to practice their writing. Many students can graduate from college without having had much practice in writing. Unfortunately, many of these students discover too late that their writing skill in the "real world"--whether it is in the form of a memo, a proposal, or a report--is the difference between success or failure in their profession.
Essays can individualize your instruction. Your feedback on an essay can help students form insights into the subject material, organizational skills, and writing skills. Your comments can also be an important motivator.
Essays provide valuable feedback to you. Sometimes objective test scores do not portray your students' learning. Essays, however, can reveal the depth and breadth of students' knowledge, as well as erroneous conclusions that are drawn.
Essay questions require less time to prepare. Good multiple-choice exams are extremely time-consuming to prepare. The essay, however, requires much less preparation. (Be careful, however, not to "throw together" a list of essay questions.)
Disadvantages of the Essay
Essays are (very) time-consuming to grade. To garner the full benefits of including essays on your tests, you must allot sufficient time to read and comment on the students' responses. Simply scanning a response and assigning it a numerical grade will not close the learning cycle. Students quickly learn how to study for your exams! If they perceive that you fail to grade their responses adequately (as indicated by your written comments and suggestions), students will not bother to study at a higher level for the next test, but will instead return to memorizing facts.
Sometimes an essay question is not representative of the content covered. An example: In an introductory course in human physiology, which has just covered blood, the circulatory system, and the immune system, seven of the ten essay questions are about the circulatory system.
If you use essays, you must use course objectives to determine the content of the questions. By referring to objectives, you will not make the mistake of focusing too narrowly on one aspect of content.
The essay exam often has grading reliability problems. Unlike the multiple-choice format, the essay is a subjective form of evaluation. And teachers often introduce biases into their grading. If you know whose paper is being evaluated, the grade may reflect personal feelings for that student, or may reflect influence from past grades. For example, if the student received an A on a past exam, you may tend to give a higher grade than is actually deserved on this exam (and vice versa if a student had received an F). Also, grades given on first papers may differ significantly from those that are graded later. Other influences include your general mood and stress level at the time of grading, the time of day, and your feelings about the subject of the essay.
Preparing an Essay Question or Exam
These suggestions may help you maximize the benefits of an essay test:
Carefully select those course objectives that can be evaluated through an essay. Cognitive objectives stressing memorization of facts, names, or definitions should not be measured by essays. Essays should require students to analyze hypothetical data, solve sample problems, or compare and contrast concepts.
Phrase the question clearly. After reading the question, the prepared student should know exactly what you expect. If a question's wording is ambiguous or too vague, unprepared students can draw upon the related knowledge they have and write an answer, while claiming (correctly) that they misunderstood what you desired.
Control the level of students' response. If you want more than just a recitation of facts in an essay, word the question so that more is demanded. Read the following examples of essay questions:
When were Medicare and Medicaid established?
How have Medicare and Medicaid contributed to the current U.S. trends of increased demand for health services, increased costs of health care and physicians' services, and longer hospital stays?
What do you suppose might have occurred in the field of health care in the U.S. if Medicare and Medicaid had never been established?
Although all three questions request information on Medicare and Medicaid, the first two require only recitation in writing an answer. The third question, however, requires a knowledge of the first two questions and requires the student to analyze rather than recite.
Write essay questions that sample the content covered. Many professors permit students to choose several essays from a list. Although you may believe this choice benefits the prepared student who may be weak in one area, actually the unprepared student benefits more from this practice because it encourages students to "place a bet" and to omit studying some key areas of content. By allowing choice, you also create different exams, which decreases the content validity and the grading reliability.
Inform students of the grading criteria. You must decide what evaluative criteria will be used. It is important to share this information with students before they write the essay. If you will be grading for language usage, spelling, and grammar, tell them. If you desire a minimum or maximum number of words, tell them. If you desire a certain format, specify it ahead of time.
Share examples of good and weak essays with students. Students can benefit greatly from seeing what is expected of them. In groups, have students apply the grading criteria to several sample essays you provide. Discuss their judgments and yours.
Ten Tips for Grading Writing and Reducing Bias
What can you do to improve the reliability of your grading of responses to essay questions? Following is a list of suggestions:
1. Cover the names of the students. Most teachers attempt to treat all students fairly. However, it is human nature to like some students more than others, or to actively dislike some students as individuals. This personal bias can affect the reliability of the grades. Grading can be influenced by the performance of the student on earlier exams, by the amount of class participation, and by the student's attitude toward the course and the teacher.
2. Familiarize yourself with the general performance level on the exam before you begin to grade. Randomly sample exams and read them thoroughly to determine the general level of performance. This practice will keep your evaluations from being unduly influenced by the quality (whether excellent or poor) of one specific paper.
3. List the points you believe should be discussed in each essay. Such a list can keep you from being "bluffed" by students who are exceptionally accomplished writers, or highly clever and verbal. Unprepared students often select one aspect of a question, or an aspect tangential to the question, and elaborate on that one point to the extent that a reader may become immersed in the writing style, overlooking the insufficiency of the answer. A list of expected points can assist you in assessing objectively the breadth and depth of the response. You can get help from your students in developing these criteria (Everett 1994). Ask them what they believe is important in the assignment, what they would expect to read if they were grading it.
4. Do not attempt to grade all exams in one sitting. As stated earlier in this article, a major disadvantage of essays is the time required to grade them. When you begin to tire of reading the responses, you naturally become too critical or too generous in your grading, which affects the reliability of the grades.
5. Grade only one question or topic at a time. Each question on an essay exam, or each essay question included in an objective test, should be graded separately. Reading an entire exam and then trying to assign a grade also affects the reliability of the grades. By grading exams by individual questions, you will be better able to concentrate on the quality of each individual response and how it compares to other students' responses.
6. Write comments on the exams. Point out the good as well as the bad. Comments indicate that you actually read the exam. Also, they serve as an explanation of why you assigned a particular grade. Comments such as "Expand," "What about . . . ," or "How do you conclude this?" readily indicate weakness to the students. When commenting, adopt a nonconfrontational attitude by assuming that you and the student share a common enterprise. Write to convince the student: be direct in your criticisms, but be dispassionate and never engage in ad hominem attack. Ask yourself whether your disagreement with the student's writing is based on specialized knowledge you possess because of your education and training. If so, then offer that knowledge to the student. Don't simply bash the student for not knowing something that "everyone knows." Maintain an appropriate perspective on the writing assignment, i.e., don't fixate on remarks that may be tangential to the principal point/s of the composition (Moore 1992). Finally, realize that you do not necessarily improve a student's writing by writing an "overwhelming" number of comments on an essay (Moore 1992).
7. Before you assign grades, sort your papers into piles. By sorting papers into piles corresponding to A's, B's, C's, etc., you can quickly check through the piles to ensure that there are no changes to be made. This practice assists you in making decisions about "borderline" papers.
8. Discuss the test with the students. Feedback is a critical element in the learning cycle. Testing provides you with an opportunity to give feedback to your students. In addition to written comments on the essay exam, you should allocate class time to discuss the question and the responses. This practice reinforces the testing process as a part of learning.
9. Have colleagues read the papers. This is an effective way of improving the grading for team-taught courses and for teachers teaching different sections of the same course. If you and a colleague differ greatly in scoring a question, reevaluate the response to the question. It is very important, of course, to tell your students about such dual grading arrangements. You could even improve the feedback to your students by having the different graders identify their comments by different colored inks.
10. If students do not meet your established criteria for a question, consider retesting them on the concepts. After providing feedback on the incorrect answers and time to restructure their concepts, allow students to rewrite their responses. Without an opportunity to revise their response, any comments you make have little effect on improving subsequent writings (see Doher 1991, for a more thorough discussion). After all, the ultimate objective of education is learning, not simply receiving a grade.
Student writing, regardless of subject, is only one-half of the equation. The other half is learning through studying the teacher's thoughtful critique of the writing. Bashing may be brought about unintentionally by a teacher's misunderstanding a student's perspective, reasoning, background, or personal values. Constructive critiquing is an art that must be learned and then practiced.
Doher, G. 1991. Do teachers' comments on students' papers help? College Teaching 39:48-54.
Everett, E. 1994. Do the write thing. The Science Teacher 61(7): 35-7.
Mayer, R. E. 1975. Information processing variables in learning to solve problems. Review of Educational Research 45:525-41.
Moore, R 1992. Writing about biology: How should we mark students' essays? Bioscene: Journal of College Biology Teaching. 18(3): 3-9.
Perkins, D. 1993. Teaching for understanding. American Educator 17(3): 8 and 28-35.
Shavelson, R J. and R Stern. 1981. Research on teachers' pedagogical thoughts, judgments, decisions, and behavior. Review of Educational Research 51:455-98.
Steele, C. W. 1992. Critique; don't bash. Writing and Learning 3(1): 5 6.
Craig W. Steele is an associate professor in the Department of Biology and Health Services at Edinboro University in Edinboro, Pennsylvania.
It’s good to regularly review the advantages and disadvantages of the most commonly used test questions and the test banks that now frequently provide them.
- Quick and easy to score, by hand or electronically
- Can be written so that they test a wide range of higher-order thinking skills
- Can cover lots of content areas on a single exam and still be answered in a class period
- Often test literacy skills: “if the student reads the question carefully, the answer is easy to recognize even if the student knows little about the subject” (p. 194)
- Provide unprepared students the opportunity to guess, and with guesses that are right, they get credit for things they don’t know
- Expose students to misinformation that can influence subsequent thinking about the content
- Take time and skill to construct (especially good questions)
- Considered to be “one of the most unreliable forms of assessment” (p. 195)
- Often written so that most of the statement is true save one small, often trivial bit of information that then makes the whole statement untrue
- Encourage guessing, and reward for correct guesses
- Quick and easy to grade
- Quick and easy to write
- Encourage students to memorize terms and details, so that their understanding of the content remains superficial
- Offer students an opportunity to demonstrate knowledge, skills, and abilities in a variety of ways
- Can be used to develop student writing skills, particularly the ability to formulate arguments supported with reasoning and evidence
- Require extensive time to grade
- Encourage use of subjective criteria when assessing answers
- If used in class, necessitate quick composition without time for planning or revision, which can result in poor-quality writing
Questions provided by test banks
- Save instructors the time and energy involved in writing test questions
- Use the terms and methods that are used in the book
- Rarely involve analysis, synthesis, application, or evaluation (cross-discipline research documents that approximately 85 percent of the questions in test banks test recall)
- Limit the scope of the exam to text content; if used extensively, may lead students to conclude that the material covered in class is unimportant and irrelevant
We tend to think that these are the only test question options, but there are some interesting variations. The article that promoted this review proposes one: Start with a question, and revise it until it can be answered with one word or a short phrase. Do not list any answer options for that single question, but attach to the exam an alphabetized list of answers. Students select answers from that list. Some of the answers provided may be used more than once, some may not be used, and there are more answers listed than questions. It’s a ratcheted-up version of matching. The approach makes the test more challenging and decreases the chance of getting an answer correct by guessing.
Remember, students do need to be introduced to any new or altered question format before they encounter it on an exam.
Editor’s note: The list of advantages and disadvantages comes in part from the article referenced here. It also cites research evidence relevant to some of these advantages and disadvantages.
Reference: McAllister, D., and Guidice, R.M. (2012). This is only a test: A machine-graded improvement to the multiple-choice and true-false examination. Teaching in Higher Education, 17 (2), 193-207.
Reprinted from The Teaching Professor, 28.3 (2014): 8. © Magna Publications. All rights reserved.
Tagged with assessing student learning, designing test questions, grading strategies, multiple-choice tests, test questions