Understanding speaking assessment: what every teacher should know

Name: Understanding speaking assessment: what every teacher should know
Uploaded: 2016-11-02T16:43:27.000Z
Duration: 1 h 48 min 38 s

Introduction

In this section, Evelina Galaczi and Nahal Khabbazbashi introduce themselves and welcome the audience to the webinar series on understanding assessment. They explain that the focus of this webinar is speaking assessment and outline the three key questions they will address.

Introducing the Webinar Series

Evelina Galaczi and Nahal Khabbazbashi introduce themselves as members of the Research and Thought Leadership Group of Cambridge English.

They welcome viewers to a series of six webinars on understanding assessment.

The focus of this webinar is speaking assessment.

The presenters will address three key questions related to speaking assessment.

The Six Key Questions

There are six key questions that need to be addressed when assessing any skill, including speaking.

The first question is "Why am I testing?" which relates to the purpose of the test.

The second question is "Who am I testing?" which relates to the test-takers who will be taking the test.

The third question is "What am I testing?" which refers to the skill being tested (the test construct).

The fourth question is "How am I testing?" which refers to the tasks in the test.

The fifth question is "How am I scoring?" which refers to how marks are assigned to performance.

Finally, "How is my test benefiting learners?" refers to the impact of tests on learning.

Three Key Questions About Speaking Assessment

In this section, the presenters focus on three key questions related to speaking assessment: "What am I testing?", "How am I testing?", and "How am I scoring?".

What Am I Testing?

The first question is about what is being tested in speaking assessment (the test construct).

Examples of constructs include fluency, accuracy, and coherence.

The presenters discuss how constructs can be broken down into sub-skills for more detailed assessment.

How Am I Testing?

The second question is about how speaking is assessed through tasks.

The presenters discuss different types of tasks, such as role plays and presentations, and how they can be used to assess different aspects of speaking ability.

How Am I Scoring?

The third question is about how marks are assigned to performance in speaking assessment.

The presenters discuss criteria for assessing speaking ability, such as pronunciation and grammar.

They also explain the use of scales to ensure that tests are reliable.

Why Is Speaking Assessment Difficult?

In this section, the presenters ask viewers why they think speaking assessment can be difficult. They summarize some of the ideas shared by viewers in the chat box.

Viewers' Ideas

Some viewers suggested that subjective marking makes it difficult to make tests fair.

Others mentioned difficulties with giving and scoring exams for teachers or examiners.

Other ideas included student confidence, anxiety/nerves, pronunciation, and using assessments in large classes.

Summary

Overall, viewers identified several challenges with speaking assessment, including subjectivity, difficulties for teachers/examiners, and student confidence/anxiety.

Introduction

The speaker introduces the topic of the webinar and outlines some questions that will be addressed.

The speaker introduces the topic of the webinar and outlines some questions that will be addressed.

Some of the questions include assessing students at different levels of ability, deciding what tasks are suitable, and dealing with student anxiety during speaking tests.

What am I testing?

The speaker discusses different types of speaking tasks and what aspects of speaking they test.

Types of Speaking Tasks

Reading aloud tasks test simple reproduction of language.

Describing differences between two pictures is a monologue task that requires test-takers to rely on their own vocabulary and grammar.

Discussing a topic in a pair or group is an interaction task that requires test-takers to interact with other learners.

Aspects of Speaking Tested

Different tasks elicit different kinds of speaking, including simple reproduction of language, monologue speech, and interaction.

When designing a speaking test, it's important to consider which aspects of speaking you want to assess.

Understanding the Constructs of Speaking

In this section, the speaker discusses the importance of understanding which aspects of speaking a task is eliciting and how different speaking tasks activate different speaking processes in our brains.

Importance of Understanding Task Constructs

It's important to ask yourself which aspects of speaking the task is actually eliciting.

None of these aspects of speaking are better or worse; it depends on the purpose of your test.

For example, a test for university entry should probably have tasks which elicit not just monologues but also interaction because interaction skills are really important for the purpose of that test.

In another context, though, the purpose of your test might be to quickly and efficiently measure the pronunciation of people applying to work in call centres. In that case, having different tasks such as reading aloud or repeating utterances may be justified.

Different Aspects of Speaking

Different speaking tasks activate different speaking processes in our brains.

Speaking is a complex activity that involves conceptualization, lexical knowledge, grammar and syntactic knowledge, articulation and monitoring.

A useful way to think about the different aspects of speaking is to compare it to driving. When we first learn to drive, we need to cover the basics before we can focus on more advanced skills. Similarly, learners need to develop core linguistic knowledge and skills before they can focus on more advanced functions like giving detailed descriptions or expressing opinions.

Introduction to Speaking Skills

This section introduces the importance of speaking skills and how they are developed.

Importance of Range of Communicative Situations

A learner needs to be exposed to a range of communicative situations, which involve monologues and interactions with others.

Different situations put different cognitive demands on the learner. Experienced speakers can deal with different speaking situations flexibly.

The CEFR level descriptors increasingly refer to the ability to be flexible and spontaneous as you go up the levels.

Eliciting Subskills in Speaking Tests

Tasks such as talking about a topic, describing a photograph, or telling a story involve monologue speech. Other tasks tap into interaction like question-answer tasks, discussion tasks, information gap activities, and role plays.

All these tasks have advantages and limitations that should be kept in mind when using them in tests or classrooms. A range of different tasks should be included to minimize task limitations.

Discussion Task Example

In this example task, two test-takers discuss what gift they could give a girl who is moving away from their tennis club. They are given some ideas to help them come up with suggestions for gifts.

Advantages: Encourages interaction between test-takers; allows for negotiation and agreement; provides opportunities for both fluency and accuracy practice.

Disadvantages: May not allow for equal participation; may require more preparation time than other types of tasks; may not elicit all subskills equally well (e.g., pronunciation).

Advantages and Limitations of Paired Tasks

This section discusses the advantages and limitations of paired tasks in a communicative classroom.

Advantages

Offers more interaction

Allows for more brainstorming

Reflects real-life speaking situations

Assesses interactional skills

Limitations

One candidate might monopolize or dominate the conversation

One student might feel too shy to talk to another student

Language ability level can affect the interaction

It's difficult to give individual scores when the conversation depends on both people.

It may be too hard to practice in a big classroom.

Addressing Limitations

Include a range of different task types, such as question-and-answer, monologue, and paired interaction tasks.

Use different kinds of assessment scales and criteria that focus on different elements of speech, including grammar, vocabulary, pronunciation, and interaction skills.

Ensure that learners in pairs are approximately at the same ability level.

Conclusion

This section summarizes the importance of using a variety of task types and assessment criteria to address the limitations of paired tasks.

A range of task types balances out each other's limitations while assessing different aspects of speaking.

Task Types and Difficulty

This section discusses the different types of speaking tasks and how to evaluate their difficulty.

Evaluating Task Difficulty

Different types of tasks have unique advantages and disadvantages.

Each type of task should be evaluated against the purpose for which it is to be used.

Two paired interaction tasks are presented, and participants are asked to decide which one is more difficult and why.

Task B is considered more difficult due to its abstract topic, requiring harder vocabulary, and a discussion element in deciding the outcome.

Importance of Topic in Task Difficulty

The activity shows how important the role of topic is in task difficulty.

Concrete topics like those found in Task A are generally easier than abstract topics like those found in Task B.

Speaking Assessment Criteria

This section discusses the criteria used to assess speaking skills.

Speaking Assessment Criteria

There are four main criteria used to assess speaking skills: fluency, accuracy, complexity, and coherence.

Fluency refers to the ability to speak smoothly without too many pauses or hesitations.

Accuracy refers to using correct grammar, vocabulary, pronunciation, and intonation.

Complexity refers to using a range of grammatical structures and vocabulary appropriate for the level being tested.

Coherence refers to organizing ideas logically so that they can be easily understood by others.

Test-Taker Preparation Strategies

This section discusses strategies for test-takers to prepare for speaking assessments.

Test-Taker Preparation Strategies

Practice speaking regularly with a partner or teacher.

Record and listen to yourself speaking to identify areas for improvement.

Familiarize yourself with the format of the test and practice answering different types of questions.

Use a variety of resources, such as textbooks, online materials, and language exchange programs.

Evaluating the Difficulty of Language Tasks

This section discusses how to evaluate the difficulty of language tasks based on their topic, type of information required, and functional language elicited. It also covers other considerations when developing or choosing tasks for language tests.

Task Difficulty Based on Topic and Type of Information Required

A task's topic and type of information required can affect its difficulty level.

Concrete topics in familiar contexts are easier than abstract topics in less familiar situations that require giving an opinion.

The Common European Framework of Reference descriptors provide guidance on the level of difficulty for different types of topics and information.

Functional Language Elicited by Tasks

Functional language elicited by tasks is another way to evaluate their difficulty level.

Informational functions like expressing opinions are generally easier than intellectual functions like persuading or taking the floor.

Task questions can be worded to elicit a wider range of speech functions, making them more difficult.

Other Considerations When Developing or Choosing Tasks

Clear instructions are important for generating the intended language output from test-takers.

Test-takers need to understand the purpose of each task so they know what kind of language output is expected.

Including a range of task types with different strengths and limitations can optimize their effectiveness in testing language proficiency.

Starting with easier tasks before moving on to more difficult ones can help warm up test-takers and build their confidence.

Timing for each task should be considered, including planning time before the task if necessary.

Clear marking criteria should be provided to test-takers.

Scoring the Speaking Test

This section discusses how to score the speaking test and what assessment criteria to use.

Assessment Criteria

The assessment criteria must explicitly state what the test aims to measure, such as fluency, pronunciation, grammar, etc.

The criteria have to reflect what the task is aiming to be testing.

Some of the criteria that could be used for evaluating a student's performance include grammar, vocabulary or lexis, pronunciation, and fluency.

Video Clips

Watch a short video clip of a student describing a photo and think about what criteria you would use to evaluate that student's performance.

Watch another video clip of two test-takers discussing a goodbye present and think about what criteria you would use to evaluate their performance.

Conclusion

This section concludes the webinar by summarizing key points discussed in previous sections.

Key Points

Assessing speaking requires clear assessment criteria that reflect what the task is aiming to test.

Fluency means how automated your speech is and can be used as an evaluation criterion.

When evaluating language proficiency in speaking tasks, it's important not to score for ideas but rather focus on language skills.

Gift Ideas for a Tennis Player

In this section, the speakers discuss gift ideas for a tennis player.

Possible Gift Ideas

A photo of her friends or a bag with her name on it to hold all her tennis gear.

A t-shirt with her name on it.

A DVD of tennis is not a good idea since she already knows how to play.

Criteria for Assessing Speaking Tasks

In this section, the speaker discusses criteria for assessing speaking tasks.

Assessment Criteria

Pronunciation, fluency, vocabulary, grammar and interactive skills are important criteria when assessing speaking tasks.

The importance of each criterion depends on what aspects of speaking you're aiming to test.

Interaction skills are essential in tasks that include interaction since they are part of the test construct.

Assessment Scales

Holistic scales give an overall mark based on general impression while analytic scales break down skills into separate features and give independent marks for each criterion.

Cambridge English has an example holistic scale while analytic scales have categories such as grammar and vocabulary, discourse management, pronunciation and interactive communication.

Advantages and Disadvantages of Assessment Scales

Analytic assessment scales can provide diagnostic information about candidate's strengths and weaknesses but only if scores are reported separately.

They're also useful in rater training since the skill which has to be measured is broken down into separate features.

They're also really useful for assessing second language learners who might have uneven profiles.

Assessing Speaking Skills

This webinar discusses the challenges of assessing speaking skills and provides strategies for creating a fair and objective assessment.

Challenges of Assessing Speaking Skills

Fluency is subjective and difficult to assess.

Analytic scales are time-consuming but offer more diagnostic information.

Holistic scales are practical but limited in providing diagnostic information beyond a single score.

Relevant aspects of speaking ability may not develop at the same rate.

Strategies for Creating a Fair Assessment

Pay attention to the test purpose and have a clear purpose.

Consider the test construct, or what you're trying to test.

Match task types to what you want to assess.

Use appropriate assessment scales and criteria.

Provide training to ensure consistent and reliable scoring by examiners.

Overall, this webinar emphasizes the importance of creating a fair and objective assessment for speaking skills. It highlights the challenges of assessing fluency, as well as strategies for creating an effective assessment that considers test purpose, construct, task types, assessment scales, and examiner training.

Exam Structure and Task Types

This section discusses the structure of exams at Cambridge and how having a script to follow can provide more information and structure. It also highlights the importance of using time wisely during monologue tasks.

Exam Structure

Examiners at Cambridge are given lots of training.

The exam is structured, and test-takers have a script to follow.

Having a bit of structure helps, but there is still some flexibility.

Task Types

Different task types exist, including monologues.

During monologues, test-takers have time to think about previous performance and what's coming up next.

Paired Tasks

This section discusses the importance of language ability in paired tasks and advises against pairing students with vastly different abilities.

Language Ability in Paired Tasks

It is crucially important for paired task success that both test-takers have similar language abilities.

If students have vastly different abilities, it is advised not to use paired tasks.

Generating Ideas for Speaking Tasks

This section provides tips on generating ideas for speaking tasks.

Generating Ideas

Providing pictures or ideas can help generate discussion topics.

Summarizing or discussing something previously read or listened to can also be helpful.

Supporting Shy Students in Speaking Tasks

This section provides tips on supporting shy students in speaking tasks.

Supporting Shy Students

Familiarity with tasks is important, so give students plenty of practice time.

Provide ideas to talk about to take the pressure off generating ideas.

Involve students in the assessment process and provide support to build confidence.

Assessing Spoken Interaction

This section discusses the importance of assessing spoken interaction and the different aspects that should be considered.

Importance of Assessment Criteria

The assessment criteria used depends on the purpose of the test.

Grammar, vocabulary, fluency, and communication are all important aspects of speaking.

For raters, grammar often stands out but communication is key.

All assessment criteria should be given equal weight.

Cambridge Approach to Speaking Tests

All aspects of speaking are equally important according to the Cambridge philosophy.

Good grammar and vocabulary are building blocks for language but not enough for communication.

Other aspects such as communicative situations are also important and given marks in tests.

Weighting of Speaking Test

In most Cambridge exams, all components have equal weighting including speaking.

Speaking contributes to the final mark as much as other papers.

Scoring in Speaking Tasks

This section discusses whether it is recommended to let test-takers know how they are being scored in speaking tasks.

Transparency in Scoring

It is highly recommended to let test-takers know how they are being scored in speaking tasks for transparency.

Clear descriptors and criteria should be provided to test-takers.

Role of Pronunciation

This section discusses whether pronunciation should be a focus when assessing spoken interaction.

Intelligibility Over Accent

The focus should be on intelligibility rather than having a native speaker accent.

Non-native speakers can still get top marks if they are fully intelligible.

Duration of Speaking Tasks

This section discusses how long pupils should be asked to speak in speaking tasks.

No Right or Wrong Answer

There is no right or wrong answer for how long pupils should be asked to speak.

Clear instructions and structure should be provided to test-takers.

Trial tests with students can help determine a good time duration.

Integrating Assessment as Part of Teaching

This section discusses how to integrate assessment as part of teaching and the importance of providing feedback during learning checks.

Frequency of Speaking Assessment

The length of time for speaking assessments should not be too precise.

Assessments should be integrated into teaching, with frequent and regular checks on speaking throughout the course.

Feedback is important during every learning check on speaking.

Dealing with Larger Classes

This section provides tips for dealing with larger classes during speaking assessments.

Tips for Assessing Large Classes

Pair up with another teacher to take over teaching while you assess groups of two or three people.

Use technology such as recordings to give groups a task and record their discussion. They can listen to each other's discussions and provide feedback, promoting learner autonomy.

Prompting During Speaking Assessment

This section discusses whether prompting is valid during speaking assessment and how it can improve adaptability in face-to-face tests.

Validity of Prompting During Speaking Assessment

Prompting is valid during face-to-face speaking assessments when students get stuck or don't understand the question.

Adaptability is key in face-to-face tests, allowing teachers to prompt students when necessary.

Students who need prompting should have it reflected in their mark.

Dealing with Unfamiliar Topics in Language Exams

In this section, the speakers discuss how to handle unfamiliar topics in language exams such as IELTS. They emphasize the importance of providing learners with ideas and support to generate language.

Providing Ideas for Learners

Cambridge English exams, including IELTS, have multiple topics that students can answer.

Even if a student is not familiar with one topic, they can still answer others.

Part two of IELTS provides prompts and structure for students to come up with something to say.

Giving learners visuals and bullet points that unpack the topic would give them support to generate ideas.

Generating Language

The focus should be on generating language rather than generating ideas.

Students should be able to produce language throughout the whole 15 minutes of the test if they are at a higher level of ability.

Conclusion

The speakers conclude the webinar by thanking their audience and inviting them to attend future webinars.

It was a privilege and pleasure for Nahal and herself to share their ideas with an international audience.

The speakers hope that their ideas were useful for their audience.

The next webinar will be on understanding test scores on November 14th and 16th.