Understanding reading comprehension assessment: what every teacher should know

Understanding Assessment: What Am I Testing?

Overview: In this webinar, Ivana Vidakovic and Dennis Carr of Cambridge English Language Assessment discuss the key ideas about reading assessments and provide practical tips. They address three questions: "What am I testing?", "How am I testing?", and "How am I scoring?". They focus on criteria for assessing responses to reading tasks, and how to make sure that the tests are fair.

Test Construct

• When creating a reading task or test, consider the following aspects of the skill to assess: level, sub-skills, time, characteristics of students (age, background, knowledge, language proficiency), characteristics of reading passages/texts, advantages and disadvantages of different task types, and scoring questions which require students to write an answer.

Real Life Reading

• When reading a timetable, consider the reading sub-skills used such as scanning and looking for specific information.

• When reading a newspaper, consider the reading sub-skills used such as skimming for main points, reading carefully for details, predicting what the text will be about, and scanning to find a particular name or figure.

• When reading a poem, consider the reading sub-skills used such as understanding the meaning of words, analyzing the structure of the poem, and interpreting the author's message.

Different Types of Reading

Overview: This section covers the different types of reading, including careful reading, fast reading, skimming, scanning, and search reading. It also provides an example task for each type of reading.

Careful Reading

• Careful reading is linear and slow, allowing the reader to arrive at accurate comprehension of the main ideas and supporting detail.

Fast Reading

• Fast reading involves skimming for gist, scanning for exact matches of specific words or phrases, and search reading which is more challenging than scanning and involves a quick search for detailed information.

Example Tasks

• Task 1: Fast reading and scanning are tested by two questions.

• Task 2: Skimming is tested by two questions.

• Task 3: Search reading is tested by two questions.

• Task 4: Careful reading is tested by two questions.

Practicing Skills

• Practicing these skills can help improve reading comprehension and accuracy.

Understanding Fast Reading

Overview: This section covers the importance of fast reading and how to introduce it to learners at different proficiency levels. It also discusses the different types of fast reading and the cognitive processes involved in reading effectively.

Types of Fast Reading

• Scanning: Searching for exact word matches in a text. Introduce this type of fast reading at B1 level on the CEFR.

• Skimming: Searching for gist in a text. Introduce this type of fast reading at B2 level on the CEFR.

• Searching for predetermined information. Introduce this type of fast reading at B2 level on the CEFR.

Cognitive Processes Involved in Reading

• Recognising letters and understanding words.

• Understanding the basic meaning of a phrase or sentence.

• Understanding implied meaning.

• Understanding meaning across sentences and paragraphs.

• Understanding information across a whole text.

• Understanding and combining information across several texts.

Other Skills Used by Skilled Readers

• Setting goals in terms of how they will read a text.

• Drawing on knowledge of the world or topic to predict what the text will be about.

• Monitoring their own comprehension as they read.

• Rereading sections of the text to check understanding if they think they may have misunderstood.

Understanding Basic Meaning of Phrases and Sentences

Overview: This section covers the basics of understanding phrases and sentences.

Jabberwocky from Lewis Carroll's, Through the Looking-Glass and What Alice Found There

• Read the next two stanzas of the poem and answer the following question: "What did the son in this poem intend to do?"

• The correct answer is C: he intended to attack the Jabberwock.

Which Cognitive Processes?

• 89% of participants answered correctly that the son intended to attack the Jabberwock.

• This task requires understanding and combining meaning across texts.

Most Challenging Task

• 87% of participants responded that task four (combining and contrasting information across several texts) was the most challenging.

Practicing Skills

• Constantly work on increasing vocabulary and grammatical knowledge.

• Teach students how to understand the same information expressed in different ways.

• Create tasks which require learners to read between the lines.

• Teach students to distinguish between main idea and supporting information.

• Teach students to identify how text is structured.

• Teach students to compare information within one text.

Encouraging Learners to Improve Their Metacognitive Skills

Overview: This section discusses how to encourage learners to be strategic about how they approach a reading task and improve their metacognitive skills.

Prediction

• Encourage learners to predict what the text will be about using the task instructions, title, or their own knowledge of the world.

• Prediction can make understanding a text easier and faster.

• Learners can use the parts of the text they have read so far to help predict what will come next.

Setting Goals

• Teach learners to set goals by asking themselves questions such as "How should I read this text, carefully, quickly, both?"

• Use an example question such as "How did the writer's attitude change during the passage?" to demonstrate how to identify clues in the questions.

Monitoring Understanding

• Emphasize the importance of monitoring their own understanding and rereading words, sentences, or sections when necessary.

• Teach learners not to spend too much time on unfamiliar words, but to identify whether the word is important or not and try to understand the gist of the whole sentence.

Testing Reading Sub-Skills

Overview: This section discusses the types of tasks and texts that can be used to elicit the various different sub-skills of reading in a test.

Text Types

• Newspapers, novels, articles, blog posts, advertisements, public signs or notices, informational material, correspondence (e-mails, text messages, letters, etc.) are all possible text types.

• The demands of a particular task and a test need to be similar to those in real life, so that we can draw parallels from how someone performs on a test to how they will perform when reading in real life.

Differences Between Text Types

• Length, style, register, target audience, lexical differences, genre, scope are some of the differences between the types of material one reads.

• Style of writing mainly refers to the difference in formality between text and the relationship between the writer and the reader.

• The kind of language used is also important, for example, comparing a doctor's language in a prescription with an academic article on a scientific process.

Text Selection and Task Types for Reading Comprehension

Overview: This study guide provides an overview of criteria for text selection and task types for reading comprehension. It covers topics such as language level, world knowledge, subject matter knowledge, language knowledge, discourse mode, text structure, and task types.

Text Selection Criteria

• Topic and Type of Information: Referring to the CEFR descriptors, topics and/or information that are personal, familiar, concrete, and not requiring specialised knowledge are easier than those which are unfamiliar and abstract or specialised and which require more difficult or specialised vocabulary and complex grammar.

• Language: Text with structurally simple and short sentences consisting of common everyday words or appropriate lower proficiency levels (A1 and A2 on the CEFR) are appropriate for lower proficiency levels. At higher proficiency levels (B2, C1, and C2), texts with structurally complex and longer sentences, including words with more complex meanings, are appropriate.

• Discourse Mode: At lower proficiency levels (A1 and A2), appropriate discourse modes are description, narration (e.g. a text telling a story of an event), and instruction. Tests set at higher levels (B2, C1, and C2) should challenge the candidates by requiring them to also read and comprehend argumentative texts (which provide views or attitudes on a subject) and expository texts (which define, explain, and exemplify a concept, term, or phenomenon).

Task Types for Reading Comprehension

• Selected-Response Tasks: These tasks require a test taker to select an answer from given options such as multiple choice, multiple matching, true false, or gapped texts where the missing word or sentences are provided.

• Semi-Productive Tasks: These tasks require some limited writing, such as a gapped text where the missing words are not given or an information transfer task may require chart or label completion or labelling.

• Productive or Integrated Skills Tasks: These tasks require a test taker to read a passage and then write a short or an extended answer, such as a summary or discussion based on that passage.

Multiple Choice Task Type

Overview: This section covers the task type of multiple choice questions, including their advantages and disadvantages.

Advantages:

• Easy to administer and mark, given that the candidates only select a response from a given set of options.

• Allow a coverage of a wide range of reading skills or task focuses.

Disadvantages:

• Require selection from a given set of options rather than writing an open-ended or semi-controlled response, thus testing only receptive skills and are open to guessing.

• Not representative of how we engage with reading tasks in real life.

Information Transfer Tasks

Overview: This section covers the task type of information transfer tasks, including their advantages and disadvantages.

Advantages:

• Come close to real life tasks in some academic or workplace contexts.

• Require comprehension of steps in a process, classification system, or comprehension of the sequence of events in a story.

Disadvantages:

• Must be presented with clear guidance for how the task should be completed.

• Need to be very carefully thought through.

Integrated Skills Tasks

Overview: This section covers the task type of integrated skills tasks, including their advantages and disadvantages.

Advantages:

• Very realistic, often in our private professional or academic lives.

• Require engaging a variety of reading skills and cognitive processes.

Disadvantages:

• None identified.

Assessing Reading Comprehension

Overview: This section discusses the various methods of assessing reading comprehension, including extended responses, multiple-choice questions, and more. It also covers considerations to keep in mind when developing or choosing tasks, such as task difficulty, clear instructions, range of tasks, order of tasks, timing, and scoring criteria.

Task Types

• Extended responses take a long time to rate and require time to develop assessment criteria and descriptors of performance.

• Reading comprehension is often assessed through writing, which can make it difficult to separate comprehension ability from writing ability.

• Different task types have different strengths and limitations, so ideally a test should use a variety of task types.

Considerations

• Are tasks at the right level of difficulty for your learners?

• Are the instructions to the task clear?

• Is there a range of tasks in the test?

• What is the order of tasks?

• What is the timing for each part of the test?

• Are the scoring or marking criteria clear to the test takers?

Scoring

• For multiple-choice questions, a correct answer would score one point and an incorrect answer would score zero.

• We can decide to award one mark or two marks, for example, to a correct answer.

• We can give every correct answer the same score, or we can use weighting to make some questions worth more than others.

• Differences in weighting should reflect differences in task demands and difficulty.

Understanding Test Scoring Criteria

Overview: This section covers the importance of understanding test scoring criteria and provides tips for teaching and practising reading comprehension.

Test Construct

• Decide on the test construct - what are you trying to test and is it appropriate for the learners?

• Select task types that are matched to what you want to assess.

• Select text with task type and test construct in mind.

Scoring Criteria

• Be clear about scoring criteria so that you and your markers can reliably score responses.

• Make sure test takers know what they're being assessed on.

Timing

• Decide on timing and communicate the timing of the overall test or of each test part, if relevant.

Key Messages

• Make sure candidates know the score for each correct answer.

• Provide information on spelling, punctuation, and grammar before the test.

• Include a range of item types in the test.

• Translation can be used to promote reading comprehension, but it is limited.

Skimming and Scanning for Language Learners

Overview: This section covers the differences between skimming and scanning, as well as tips for encouraging extended reading.

Skimming and Scanning

• Skimming is a form of selective reading that focuses on getting the gist of the text, while scanning is more about searching for exact words or phrases.

• When learning a new language, it is important to move away from the mediation of your mother tongue and make the process as automatic and spontaneous as possible.

• It is interesting to consider how the decrease in reading among millennials is affecting reading skills.

• Technology can be incorporated into assessment, such as computer-based assessment.

Encouraging Extended Reading

• Having a good library at school or language centre is key to encouraging extended reading.

• Genres and text types should be varied to keep readers engaged.

• Simplified readers are a good entry point for lower levels, but should be encouraged to move on to original texts.

• Teachers should think outside the box and use authentic materials, such as street signs or notes, for lower-level readers.

Accessing Authentic Material for Teaching

Overview: This section discusses the availability of authentic material for teaching and how to access it.

Finding Authentic Material

• When starting out, the choices for authentic material were limited.

• Now, there is a lot of material available.

Conclusion

• If you have any further queries related to exams, please drop us a line and we'll be happy to respond.

• Thanks a lot. Take care now.