Critical Evaluation Of Two Articles That Describe The Inconsistencies In Rater Behaviors

1267 words - 6 pages

Rating an essay is considered a complicated task for raters to remain consistent in their decision making process (DMP) when rating. It is because different raters have different qualities which affect raters’ behavior when assessing writing. Several researchers describe these qualities as raters’ proficiency level, raters’ experiences and tasks as the cause of raters’ inconsistencies in raters’ performance in their studies.Other researchers in recent studies suggest that rating process and rater cognitive process are the main cause of these variabilities.Since the level of variabilities determines both the score validity and the rater’s reliability, it is necessary to identify other causes of rater inconsistencies in the DMP.

The purpose of this paper is to critically review two articles that describe the inconsistencies in rater behaviors related to the factors studied. Khaled Barkaoui (2010), in his article “Variability in ESL Essay Rating Processes: The Role of the Rating Scale and Rater Experience”, identifies effects of rating scales and experiences on raters behavior through think aloud protocol. Beverly Anne Baker (2012), in her article ”Individual Differences in Rater Decision-Making Style: An Exploratory Mixed-Methods Study”,states that decision making style (DMS) is related to the DMP in writing assessment rating.The following critical evaluation will be presented first with a summary of the two articles and its evaluation will be presented later with each articles strength and weaknesses ,prior to the conclusion of this article.

Summary of text one
Barkaoui’s (2010) quantitative study of variability in raters assessments discusses the impact of rating scales and rater experiences in essay rating context (p. 54).In this study, 25 raters assessed 12 essays silently and 12 while thinking aloud. Each raters were identified as either novice or experience raters with at least five years experience (Barkaoui, 2010, p. 56). After rating holistically and analytically, the rating scores were collected. Results show that rating scales have greater impact on rating processes, especially in analytical ratings( Barkaoui, 2010, p. 56). Although novice raters rely more on rating scales than experience raters, the author contends that the effects of raters experiences are not evident ( Barkaoui, 2010, p. 61). The author concluded that essay raters and researchers should consider rating scale a greater effect than rater experience in a rater DMP ( Barkaoui,2010, p. 66).

Summary of text two
Baker (2012) focuses on rater decision making style (DMS) and rater profiles frequencies that affect the DMP ( p. 228). Rating scores and comments of six different raters were collected during the English Exam for Teacher Certification (EETC) by ‘Write aloud’ protocol and general decision making style inventory (GDMSI) questionnaires ( Baker, 2012, p. 227). Baker assumes that the study will provide an insight to individual rater cognition ,enabling...

Find Another Essay On Critical Evaluation of Two Articles that Describe the Inconsistencies in Rater Behaviors

A Critical Evaluation Of The Representation Of Crime And The Agencies That Deal With It In John Grisham’s “A Time To Kill”

4313 words - 17 pages A Critical Evaluation Of The Representation Of Crime And The Agencies That Deal With It In John Grisham's "A Time To Kill" A Time To Kill was John Grisham's first novel, and by his own admission there is "a lot of autobiography" in it. This legal-procedural novel follows the trial of a black father, accused of murdering the two white youths who raped his ten-year-old daughter. The main protagonist is not the defendant, Carl Lee Hailey, but the

This Analyzes Two Works That Describe Ancient Rome From A Philosophical Point Of View

786 words - 4 pages big as Pliny would suggest with this, but he does admit that it is only "26 miles" if you draw "a straight line from the mile column placed at the entrance to the forum to each of the gates, which are at a present thirty-seven in number", and it is a circle because Pliny tells the measurement of the "circumference" of the city (Pliny). Pliny gives great details that really describe what the city is proud of. One achievement they are particularly

Critical Analysis: Comparison/Contrast Of Amy Tan's "Two Kinds" And Flannery O'Connor's "Everything That Rises Must Converge"

643 words - 3 pages Most children rebel against their mothers at some point in time.The characters in Amy Tan's "Two kinds" and Flannery O'Conner's "Everything That Rises Must Converge" were similar in this aspect. Although the way felt about their mothers, the reasons they rebelled, and the way they went about it are very different. On the other hand, in the end of the stories, both characters had similar feeling towards their mothers.How Ni-Kan in "Two kinds" and

Often, Pop Music Is Separated Into Two Groups: That Which Is Standardized And Contrived And That Which Is What Theodor W. Adorno Would Describe As "Serious"

1864 words - 8 pages Popular music has long since dictated what the public listens to. Often, it is separated into two groups: that which is standardized and contrived, often a by-product of the music industry, and that which is what Theodor W. Adorno would describe as "serious". Two such popular artists exist today, Avril Lavigne representing the former and Weezer, the latter. To support this, Adorno's arguments on the state of popular music can be implemented

Explainations of Research Methods in Two Psychology Articles

1153 words - 5 pages observations is that they are time consuming as the researchers might have to spend time in observing behaviour before they can gather enough evidence to support their claims. AC 3 After explaining the research methods of the two articles, I am now going to evaluate the ethical issues involved in both researches. One of the ethical issues of Harlow’s research is animal cruelty (Cherry, ND). During the course of the experiments, Harlow had

"The Merchant of Venice": Critical Evaluation

1142 words - 5 pages shows a warped sense of kindness when he does so. He tells Antonio that if he cannot pay it back within the month, then he must give Shylock a pound of flesh. He shows brains in doing this and Antonio shows naivety in agreeing to this. Shylock also proves his quick wittedness and business brain in Act 1 Scene 3 Lines 121 and 122."You called me dog; and for these courtesies i'll lend you thus moneys."This quote shows that even though Shylock hates

Comparison of Two Newspaper Articles

1999 words - 8 pages Comparison of Two Newspaper Articles These two articles are about asylum seekers. Both articles agree that there are too many asylum seekers, but are presented in a very different way to each other. Article one is a shorter more aggressive tract, not intended to make the reader think about the article but just to instantly agree with it. Article two however stands back and looks at the problem, why it has arisen and

Describe the advantages and disadvantages of living in a world such as that depicted in "Gattaca"

865 words - 3 pages Describe the advantages and disadvantages of living in a world such as that depicted in GATTACA. Secondly, imagine that you live in this world. Describe how you envisage your life, including the good and bad. Include the kinds of issues that people living in this world would have to deal with everyday.The film Gattaca is based on the theme of a eugenic society in which science and medicine are imperative in the running of the world. Medicine in

GCSE media studies coursework: A comparison of two local newspaper articles. This essay compares two newspaper articles taken from two different local newspapers in England

1264 words - 5 pages In this essay I will be comparing two articles taken from local newspapers in different areas. I will be looking at the techniques used in each article and their effectiveness. I will use this comparison to eventually decide which article is the more effective of the two.The two headlines from the articles are quite different. 'Inches from tragedy' is a large, bold headline written in a way that makes you want to see what the headline is talking

Describe the components of fitness that are significant in this movement performance and sport. - PDHPE - Assignment

225 words - 1 page 100-metre Sprint The objective of the sprint is to propel the body down the track as fast as possible. Muscular Strength is the ability apply force against a resistance in a single maximal effort. Muscular Strength is used in a 100-metre when a sprinter is starting off the starting block using their legs to push themselves off for a run. Reaction time is the time taken to respond to a stimulus. This is important for a 100-metre since you need

A Comparison of Two Newspaper Articles

1682 words - 7 pages to eventually decide which article is the more effective of the two. The two headlines from the articles are quite different. 'Inches from tragedy' is a large, bold headline written in a way that makes you want to see what the headline is talking about, as it sounds serious. 'Firemen free bathroom tot' on the other hand is quite small for a headline and only really uses one emotive word, free, making it sound like the

Similar Essays

Critical Analysis Of Two Articles That Attempt To Attribute To Certain Human Behaviors Psychology Of Social Behavior Assignment

435 words - 2 pages Article #14 Question #1). Table 1 depicts certain behaviors or attributes that have historically been either male or female dominated. While it is stated that this table was put together in 1974, I feel that to an extent they are still applicable today. Although we have made great strides in gender equality, some might argue you cannot go against genetics. Historically, women have been more tender and affectionate due to being the caretaker in

Critical Review Of Two Articles With The Concept Of Interpellation

1747 words - 7 pages When creating the title for their book, which is dedicated to Stuart Hall, perhaps Paul Gilroy, Lawrence Grossberg and Angela McRobbie did not know that they not only found an exact citation to summarise all the works of this influencing cultural theorists, but also proposed the best phrase to describe Cultural Studies, that is "without guarantee". Indeed, Cultural Studies devotes itself to questioning knowledge "guaranteed" to be true in

A Critical Evaluation Of One Of The Theoretical Approaches Used To Describe Pattern/Object Recognition

1206 words - 5 pages essentially involves a hierarchical system of demons that perform various information processing tasks that eventually lead to the recognition of an image. The pandemonium model involves several simultaneous stages compared to template models, which only involve two stages and looks at more detail when analysing what the particular pattern/object is. It also has the advantage over template models in that with a limited set of feature detectors it can

Taxation; Find Two Articles That Discuss The Local, State, Or Federal Taxation Of A Good

710 words - 3 pages tax receipts are often dedicated to transportation projects so that the fuel tax is considered by many a consumer fee. Because of the relatively inelastic nature of demand for fuel, in the short run the tax will be an effective source of revenue. In the long run, however, the demand may be more elastic, since people adjust their consumption of fuel. Over a period of years, people will consume less as the price increases by switching to more fuel