In Episode 4 of The TOEFL Podcast, Paul Austin of TOEFL Speaking Teacher and Roman Libov, a former TOEFL Speaking evaluator discuss the TOEFL evaluation process and how you can perform better on the TOEFL speaking section.
Roman has been teaching English for about 7 years all over the world. He found, that in every country, every place he lived in students were struggling with the TOEFL. Roman wanted to learn more about how he can help students and decided to become an evaluator. And as a result of that experience, he was able to become a better TOEFL instructor.
Roman thinks that a large part of the exam is very mechanical, there are specific things that the ETS is looking for, when they are grading a speaking response, and those particular elements are all listed in the Independent and Integrated Speaking Rubrics. Roman emphasizes that students should become very familiar with these rubrics. In this Podcast Episode, Roman focuses on the Speaking Rubrics, reads the Rubrics examples and gives his explanation on each of them.
Very often students believe that they can crack the exam just by knowing how to answer the questions. Roman thinks the language production is not that something you can crack based on memorizing of responses that you´ve heard before. Students have to be able to produce English well regarding a lot of factors.
Most students need 26 -27 on the speaking section, and so they ideally need to be able to have a great Topic Development, Language Use and Delivery.
Roman explains what are effective ways to make sure that when you are giving responses, you can fulfill all those criteria that they are good enough to be able to get a score 4. A 4 does not mean it´s a perfect response; it just means that it´s at a fluent level, and that is above the average of what they´ve rated in terms of overall of students responses.
As TOEFL instructor, Roman speaks about a different set of skills, that he is teaching students for TOEFL preparation and when he is just teaching general English.
In this episode, Roman tells us about methodologies and strategies he uses in the TOEFL classroom to help students get high scores in speaking section.
Check out this episode!
You can find TOEFL Speaking Rubrics here
Hello, TOEFL students and welcome back to the TOEFL Podcast! We have a very special guest for you today, his name is Roman Libov, and he is a formal TOEFL evaluator for the speaking section specifically, and we brought him here so that you can get kind of an inside look as to how you can perform better on the TOEFL Speaking section. Thank you, Roman, for joining us. It´s a pleasure to have you.
Roman: Hello, Paul, thank you for having me on.
Paul: So now we were talking a little bit before this, we spent 30 minutes and got to know each other pretty well. But can you tell our audience a little bit more about, you know, who you are, what is your experience in teaching English and how did you become an evaluator?
- Sure, sure. I´ve been teaching English for about 6 to 7 years. It´s about 7 years now, and I´ve taught all over the world in about 73 countries. And when I got back to the States, I thought, “Hey, I really would love to help students in terms of the TOEFL,” because I found that pretty much in every country, every place I lived in, students were struggling with this particular exam. So, I wanted to learn more about how I can help students, and I decided to become an evaluator. So I did that for a while.
P.: And how was that like? How did you find the evaluation process, was that something that you enjoyed or was it something that was difficult and challenging at times?
R.: I would say, it was a combination of both. I would say, most of the time I enjoyed it. There were times where it sounds very, almost mechanical, and so I preferred teaching to evaluating. But I would say that the evaluation is. When I was doing that, I though when I was doing that, really helped me become a better TOEFL instructor. And as a result of that experience, I was able to help quite a few students do better, do much better in their exams.
P.: So what did you learn from there? Like what type of methodologies, what type of strategies did you use in the TOEFL classroom to help students get high scores either in speaking section or all sections.
R.: Sure, sure. I would say, in terms of the actually the evaluation experience, you just really learn what the test is about. So, you know, you basically learn that elements of the exam, a large part of the exam is very mechanical in the sense of, there are specific things that the ETS is looking for when they are grading a speaking or writing response, and those specific elements are all listed in the Independent and Integrated Speaking Rubrics. And so I always told students, if they are trying to get a high score on speaking section on the exam, or the writing section and they should become very very familiar with the rubrics which are available for free online in a bunch of different sites including ETS site.
So that is the mechanical piece, the specific things they are looking for. However, I would also say that the other half for particularly speaking or writing section is that your English does actually have to be on a certain level to score at a certain percentage. And often students think that they be very relaxed. You know, they can study and they can crack the exam just by knowing how to answer the questions. That´s actually false, because students actually have to be able to, you know, produce English pretty well in terms of a lot of factors: grammar, in the writing section, spelling as well, syntax, the way you put sentences. It is a lot of things that are really really important. Actually, the language production and it´s not that something you can crack based on memorizing, you know, responses that you´ve heard before, particularly for the independent section of the exam, I would say.
P.: And that´s there is a lot that you just spoke about. So I gonna back. It sounds like, if you are talking about the rubric, it´s often times, I kind of break it down into this with my student as well, you know, I say there are specific skills you need to master for TOEFL exam, like note-taking, organization, the ability to deliver a response in a time limit and then there are also certain things that you just need to do well from an English speaking perspective so like you are talking about, being accurate with grammar, being able to speak with confidence, have a pretty good level of fluency, some of these things. So, could you.. let´s go back through every one of those specific skills and the rubric. Will you maybe summarize some of the things that are in the rubric?
R.: Sure. Absolutely. If students have a chance to download the rubrics, I would say, that´s the first step and they should download the rubrics and they download both the speaking and the writing rubrics.
Today we are going to focus on the speaking rubrics. So, there are two rubrics, the Independent Speaking Rubric and the Integrated Speaking Rubric. If you look at the two rubrics side by side, they seem to be very similar. There are definitely a lot of similarities, but in specific areas, there are lots of differences. And you can see those by looking at them both. But if we have looked, for example, the Independent Speaking Rubric we look at, we’ll start with the top score and we compare it to a score of, let’s say 2 because I think that´s a good range for most students. So if you look at the top score, a score of a 4, there is a general description, I I’ll just read it, I have it here, it says, “The response fulfills the demands of the task, with at most minor lapses incompleteness. It´s highly intelligible and exhibits sustained, coherent discourse.”
Here is the key part, “A response at this level is characterized by all of the following.” And then you´ll see three more sections, where one is Delivery, one is Language Use and one is Topic Development. So, when it says, “It´s characterized by all of the following” which means it has to fit all of the requirements for this particular score which means Delivery, Language, Use and Topic Development.
And just to compare, if we jump down to 3 it says, “A response at this level is characterized by at least two of the following.” That means it can either be Delivery plus Language Use, Language Use plus Topic Development. So could be, or it would have to be two of those, and as you move down the rubric, you´ve noticed that it less and less.
That is, I would say a sort of a broad perspective, this is really really important for understanding. And we could focus on each section, you know, as you like.
P.. So, yeah, let´s talk more about each section. For most students, at least a lof of students I worked with, they need 26 -27 on the speaking section, and so they ideally need to be able to have a great Topic Development, Language Use and what´s the third one?
P.: In terms of what you noticed either as an evaluator or as a teacher, what are effective ways to make sure that when you are actually giving responses, you can fulfill all those criteria that they are good enough to be able to get a score 4?
R.: Absolutely. Let´s look at the score 4 then. And we´ll look at Delivery here. So, I´ll read it and I´ll provide some more expansion on this. Generally, if we look at delivery at a score 4, the example provided is “Generally well-paced flow ( fluid expression). Speech is clear. It may include minor lapses or minor difficulties with pronunciation or intonation.” What does this mean? This does not mean you have to sound like a native speaker. It just means that generally it has to flow and be understood, and you can have minor lapses. Even native speakers have minor lapses. I´ve probably had several during this podcast, where I might say, ” amm ” or when I pause, we call that sort of natural speech so minor difficulties might be with pronunciation, maybe instead of saying “example” you said “exaimple, “there might be some other minor intonation issues. However, what this means it doesn´t take away from your meaning or what you are trying to say.
So Delivery, even though there might be pronunciation issues or minor lapses or intonation issues, it does not take away from the meaning of what you are trying to express. As soon as either the pronunciation or the intonation or the lapses take away from the meaning of what is trying to be expressed, and the evaluator cannot understand what that particular sentence was about, and you score is likely to be downgraded for that particular section of Delivery to a 3.
However, if you hit a 4 in the Language Use, Topic Development and you get a 3 on the Delivery, you could still potentially hit that format for the Independent Speaking. It just depends, because I just remember, it depends on how you do. However, unlikely if you really hit the Delivery and it´s really was misunderstood, more than one sentence, it´s unlikely that you jump up to probably down to 3, because then you´ve hit only two of the following, just the requirements for the 3.
So just depends, and that´s why you usually have two or more evaluators with you for each response because there might be some disagreement. Because if just one sentence there was misunderstood, the overall meaning for like In Independent response in the 45 seconds response is still very clear and well organized in a structure, you can still get a 4.
A 4 does not mean it´s a perfect English response, a perfect response in English language, it just means that it´s at a fluent level and that is above the average of what they´ve rated in terms of overall of students responses.
P.: And so, knowing, you know, when you were evaluator, when you taught in the classroom, how did you maybe adjust, how you work with students to ensure you can do the most effective use the most effective methodologies to make sure that if they had a chance of getting to get a 4 they could to have 4 for example, on Delivery or Topic Development or Language Use.
R.: I think particularly for Delivery, and I´ll jump to Language Use as well, for Delivery, one of the best things students can do is for example, if you watch a television show, if you see if we think of some sort of trashy American television, you will generally see delivery done in a very natural way. If you´ve ever watched an interview, if you are listening to this conversation, for example, our podcast, delivery is in a very sort of natural way. There is a well-paced flow, the speech is clear, there are some minor lapses, less so minor difficulties with pronunciation but there are minor lapses, so an overall where understood might be some nuances.
I would say things like as, you know, things like podcast, you know, trashy American television, anything like that will indicate students what delivery looks like. So let´s say, you know, radio, podcasts, television shows, movies, these are really great ways for understanding Delivery. It´s a little bit difficult to read that unless you are reading through a sort of a fluent lens or you are reading through a native speakers point of view.
In terms of language use, again, I´m gonna focus on particular parts here and the language use for score 4. There is a term that uses “Effective use of grammar and vocabulary.” Well, this is a little subjective, because what does that mean? From ETS perspective, from the training of the evaluators, this means that, again, the grammar is used generally, accurately and the meaning is portrayed in a way that represents the way that the speaker wants to. In another words, the evaluator understands what the speaker is trying to say whether they use past, they use present, they use Past Perfect, they use Present Perfect, they use future, they use Future Perfect. Did it make sense for that particular grammar usage? Another words, if you are using the past, are you talking about the past or are you using the past talking about the present?
So if you are using grammar correctly, it could be something like using, you know, Present Simple correctly or Present Continuous correctly. If you used Present Continuous incorrectly twice or, you know, this is taking away from the meaning and the language use is affected by that point. So, it´s very important that students, this is where it´s almost the critical piece. Because this is where is clear if students don´t have a full grasp of English grammar.
This particular section is particularly important. Because again, if it takes away from the meaning, if you are talking about the present, but you really mean to talk about the past and that´s gonna affect the score. The other part here is that it exhibits a very high degree of automaticity with good control of basic and complex structures. So, this is again, what does it mean, a “good control of basic or complex structures “? This does not mean you have to use the Future Perfect, it just means that you have a range of grammar structures and that could be any range. It does not necessarily mean the specific grammar structures in mind. So students often make mistakes, in this particular element, this particular piece of rubric, oh I have to use the Future Perfect, the Present Perfect or the Present Perfect Continuos, that’s not what it´s saying, there are no indicators even for the raters where it says, they have to use specific grammar rules, there are no tenses and aspects, there are no rules, that say that, it just means a range.
P.: And so, when you were evaluating, what would you consider like a decent range, would it be 2 or 3?
R.: It just depends on the range, for example, if the student´s Delivery is clear and they are using kind of simple language, but it´s clear, there are no mistakes, I might say, “Well, considering the Delivery, the Topic Development, they delivered a good, you know, good structure organization to their actual response, I may look less on the Language Use.” You know, I still think about it, unless you are making mistakes, in structure, I might say, “Well, that´s still considered a range of, you know, a very high degree of , there is an automatic speech, also that control of complex structures. It just depends on a response, and that´s why there is some subjectivity, that´s why you have two to three evaluators. It just depends, there is a lot of factors to be considered and how much Topic Development. The Topic Development is so thrown, there is likely a chance that they use a range of structure and that might mean present, past maybe they´ve used Past Perfect, maybe they´ve used the future. It´s unlikely that someone has a lot of topic development, and they haven´t used complex structures. That´s rare. I´ve probably come across that very barely that it was fully developed just using the Present Simple, it just depends.
P.: Now, let´s kinda hit the other side, when you are in the classroom, and you are trying to develop a students language use, so that you know, they are able to flexibly use maybe two or three different tenses with their response comfortably, what sort of exercises or how would you approach that?
R.: Sure. I would say, when I am teaching the TOEFL it´s a completely different sort of instructions, as an instructor, a teacher versus just teaching English. I actually recommend, that students, I don´t usually recommend students just take the TOEFL when they first starting to learn English. And when I say “first starting “, I mean the first six months to a year, and students often don´t listen to me and still take the test and they get, you know, lower score that they need, because their English development is still quite low and they haven´t have the time to really grow in terms of language development. And so I would generally say in the classroom if I am teaching English I don´t necessarily ask students to go and use complex responses, we might be focused on a particular tense or particular type of skill in terms of English, it just depends. But if I am teaching TOEFL, I have someone, you know, they are making mistakes, I don´t know, Past Perfect, and they are trying to use it and they are making mistakes, I just correct the Past Perfect, like “Did you notice you are doing this? ” Just focusing on one skill or grammar point. So just depends.
I would say as TOEFL instructure, for me I find it´s a different set of skills, that I am teaching students and when I am just teaching general English. But at the same time, if I am teaching the students who say, “OK, I´ve been English learning for a year, when I wonna get 110 on the TOEFL,” I would say, “Good luck, but it´s very difficult to do that”. Because the language development isn´t there. It takes longer to really develop fluency in English to be able to score that high on the TOEFL. So it just depends on a type of students that I have.
P.: And I think that´s an important point you emphasize, you know, in one of the previous podcasts, the last podcast that I did actually with a woman named Marina, who got like a 117 out 120 on TOEFL. For her that was achievable because she has basically been learning English since she was 5, and I think in a very prestigious school, she was basically like a native speaker. And I think everyone, when they are taking the TOEFL or when they are learning English, they have to realize that they are on their own journey, and the journey that they´ve been on through on their language learning is different from the other people. And many other people have maybe spent more time or even spend less time, on establishing like, we are talking about the foundation of Language Use and Language Development, and some people just need to go through the process of getting more exposure to spoken English, and getting more practice speaking English, and just learning a little bit about the nuances of English learning, which are like a little details of English learning. So, I think that is a very good point to emphasize, is it´s important to have an established foundation of language use, if you are expecting to get a high score, and I have a high number of students, who that should be their own priority, shouldn´t be on trying to crack the TOEFL, but it should be on just getting more language use practice.
R.: Absolutely. I agree with what you’ve just said 100 percent. And I would say, for me, as someone who was a ESL student myself, you know, English is my second language, there is something that happens for Oxford learning a new language after a certain age, where they blend the two languages, and that can happen very often. And the problem is with the TOEFL, it doesn´t fully accept that always in their training materials, even if they are from the ETS. I would say for students who are, you know, second or third of fourth language learners, it´s really important to understand these rubrics. Because when you are responding for example to an independent speaking question, it´s really important that you are not blending multiple languages, you really kind of focus on what they are asking you to do in these rubrics. And clearly, language does not always work in terms of how these rubrics present in terms, you know, what is a good response. But for the TOEFL, because it is a standardized exam, these rubrics, I would say, are very important for ESL and ETFL learners, so that would be my just one thing to emphasize the real importance of these rubrics.
And as a rater, as an evaluator, I´ll tell you this, we don´t get much more information that what students see in terms of these rubrics. There is a bit more information that about the criteria that is, you know, when they tell us how, what should be looking for for this particular question, depending on the question. Again, that´s not, that´s only priority, because they don´t want students to answer the same questions, but in terms of what the raters are looking for we have a very similar rubric and how we are gonna evaluate this exam, particular for the speaking section.
P.: I think, that´s important to have that transparency so that it is clear what the student need to do when they are practicing. And this bring me back to the point when there is lot of students who think they can get lucky, they can just go and take the test and see what happens, we talked about this earlier, when we spoke about that before the podcast as well, you can´t really get lucky on the TOEFL, right? You cannot crack it, you need.. most people who supposedly get lucky who supposedly crack it, it´s only because they spent years and years and years of studying English before it.
P.: So I think, that´s important to emphasize. I wonna hit the last point as well in terms of Topic Development, what might be a difference between a 4 and a 3 ? What was your experience with that?
R.. So and again, I just gonna read this, if you are just looking at this for you, know, Paul will provide the links to these rubrics after your podcast. So I´ll read it out loud just, “Response is sustained and sufficient to the task. It´s generally well developed and coherent, relationship between ideas are clear (or clear progression of ideas). So, the way that I would teach and the way I´ve taught TOEFL, the way I´ve generally taught. Again, we just focus on the independent speaking rubric. I always teach students this particular question, independent either the choice or the preference task is quite similar to the essay task, to one of the essay tasks, in the TOEFL when you have, you know, a thesis or you have ideas to support your thesis. The only difference is that is 45 seconds versus having time, you know, for writing a response and edit and all that, it that´s a very similar to what I teach, very similar response. So I use the deductive method “Tell me what you gonna tell me, tell me, tell me what you told me”.
Some people use inductive approach when they tell a story and make a conclusion and that´s OK. I just prefer the deductive piece, it worked better for me, as a rater I like to see that, however not all of the raters are gonna do the same, some people do it differently. But the key is if you use this sort of “tell me what you gonna tell me, tell me and tell me what you have told me”, you generally provide a well-developed piece and the relationships are clear.
Whether it´s coherent, it´s a whole different thing, and that depends on how good your actual language production is, and do your examples support your thesis and does it help you develop your story, to support, you know, what the thesis is trying to argue. So I would say, this particular topic development I would say the core of it is the structure and organization. The actual topic what you are actually gonna speaking about as providing a long enough response. That means for the Independent is 45 seconds of actual speaking. For the Integrated task, it´s gonna be 60 seconds of speaking. That means just providing enough information. And if that means that you are on your conclusion statement, and you get cut off that´s fine, you´ve already presented your thesis and your two points to support your thesis. And again, in the speaking, I generally recommend either 1 or 2, not 3 like I would in the writing for the independent essay, because you just don´t have enough time.
And often I would recommend that students choose one response, one development piece to the pieces. Because then you can spend the full time on the developing your idea so for example if there is a question like “Do you believe zoos are good?” You say, “Yes, I think zoos are good for educational purposes.” And that is your first point. Develop that with your example, really develop why you think that, based on specific personal examples, why you think in that is a case. And that would be Topic Development, really filling that 45 seconds with the clear thesis, clear body and then a final concluding remark. If you get cut in the concluding remark, is less important then if you get cut off on, you know, in the development piece.
P.: Roman, I´d love to keep talking, but I think our viewers, our listeners I would call them, are there is a lot you´ve just helped with talked about. I wanna thank you first of all for coming on the TOEFL podcast. I am just gonna briefly review what we spoke about.
We were mostly just looking the rubrics in terms of Language Use, in terms of Delivery, in terms of Topic Development, what those expectations are for the test takers. And how, you know, actually in the classroom or whether working in their home, they can actually know they are doing the right things to be able to deliver a response, that could potentially earn 26 or 27 or 28 on the TOEFL speaking section. And I think, the advice that you´ve provided was great, it´s great to have your perspective as both as ex- evaluator and also a former teacher. Thank you so much for coming on the TOEFL Podcast. I really appreciate you´ve been here.
R.: Thank you for having me here, Paul.