TEFLChina.org > Teaching >
Speaking:
Speaking Exams: a compiled discussion, 24 May 2001 through 7 May 2003
--------------------------------------------------------
From: Jennifer Wallace
To: teflchina (at) yahoogroups.com
Sent: Thursday, May 24, 2001 3:54 PM
Subject: Re: (teach) Oral English classes: Giving grades
In Britain I used to work for Cambridge (UCLES) and last semester gave my students a slightly simplified version of the lowest level of the suite of UCLES exams - and it was very successful Good ones get a chance to shine, poor ones can get through. You can tell them EVERY question (etc) beforehand and it still sorts them out. It also took only about 5 minutes per person. I taped the exams and marked them fterwards. E-mail me direct and I'll send you the 'script' I used (COMPILER'S NOTE: see further down, where it was sent to the list).
------------------------------------------------------------
From: "Ryan Schreck" schreck517 (at) hotmail.com
Date: Sun May 27, 2001
Subject: Re: Oral English Class: Giving Grades
Dear Nancy,
Last term I made the same mistake as you - inadvertantly penalizing the shy students. I became aware of this problem (and corrected it) by giving more oral exams this term. I do this by cancelling class for the week and instead have them come to my apartment to meet with me. My classes aren't too large, about 25 kids, so 90 minutes/week is enough time for each student to come talk to me one on one for 4 -5 minutes. I give them a very general topic - tell me about your last vacation, something in the news, or just how they are doing in general (now that I know them better they always talk about their boyfriends). I try to give them the opportunity to talk about something they are interested in and familiar with to prevent too much memorization. I have done this three times, and the final exam will make four. As for grades, I found an oral grading scale in a book (I'll have to look for it at home if you want the name). Basically, it is a ten point scale - 5 points for accuracy and 5 points for fluency. Accuracy consists of vocabulary, pronunciation, grammar, and accent. The fluency score is based on how well they communicate in general. This score is usually the higher of the two. I find this scale helps me to avoid weighing one student against another and I remain more unbiased. Now that I have a number of oral exam scores, they will account for a large part of the final grade. Attendance will be minor. Participation will be the deciding factor. Somebody with poor scores but great participation will get a higher score because she has shown a desire to improve, and vice versa. Because oral english demands practice, those that do practice certainly deserve a higher grade for their efforts, even if they have nothing else to show for it. So, the shy students do lose a few points in the end, but because I have met with them frequently I have a much better idea of their ability, which will cast a large reflection on their final grades.
Ryan
---------------------------------
From: "Jennifer Wallace" Jennifer0052 (at) hotmail.com
Date: Mon Jun 4, 2001
Subject: Oral English exams
Several people asked me for a copy of the script I've used for oral exams, based on past experience of being an oral examiner in the UK for Cambridge (for KET and PET). I can't respond to the request at the moment as the material's on my laptop which is about to go off for repair. I'm downloading stuff to floppy discs (with problems) so will try and put the stuff on the list in the next week or so.
Some back ground on this is ...
All Cambridge's oral exams are now done with 2 examiners - one just listens, the other does the interaction, giving instructuions etc. Both give marks, but the one interacting only gives an overall impression type of mark - you can't do better than that if you're also interacting. Here I'm getting round that by recording all my exams, so I make no attempt to mark until afterwards. One important thing is to say the name of the student at the beginning of their recording!
Secondly, as a native speaker you don't need a large sample to make a good assessment of a speaker's level - you do it fast and well on a surprisingly small sample. Often KET and PET exams are as little as 5 minutes, some of which will be you, the examiner speaking, and some of which will be silence while the student is thinking. I think coursewook/classwork is the time for lengthy performance on the part of the student, not in exams, which will drive you demented if they go on too long (I have around 200 students to examine).
I've done my orals here at the end of the college/university student's first semester as individual exams. At the end of the second semester they'll be in groups of 3, talking with each other with minumal input from me, and big penalties for every word of Chinese used. It'll be fair because I've have trained them for what they're going to be doing in the exam. (They've already heard recordings of themselves discussing things as well as for pronunciation work.)
For their first oral exam I even told them what the questions I'd ask would be, e.g. What was your favourite subject at school? Why? What's wonderful is that even when they know, there's still a real range in performance. You can see who's worked hard, prepared and rehearsed and who then comes out with something reasonably fluent just from recognising my key words, and you still get those who stumble along.
Hope this helps - I'm about to start oral exams here in the next 2 weeks.
---------------------------------------------
From: Katy Miller katyinchina (at) yahoo.com
Date: Mon Jun 4, 2001
Subject: Re: (teach) Re: small groups in class/role plays
I had fun with a role play activity the other day. Recently the students have been given an extra class each week - well, it's not really a class, I was just told to "supervise their reading" for one period a week. So I thought it would be a good chance to get them reading current news articles.
In the oral English class immediately following that period, we did "TV English news". Each group had twenty minutes to prepare a news item, with interviews and people acting out scenes in the background. They were really very open-ended - I just said "make a TV news item", and they were fantastic. One group did something on the environment, another did an item about the spring outing we were taking that weekend (belated), another group did one of the news items they'd read in the reading session, etc. I was the anchor woman.
It was great fun and I was impressed with the job they did. One group even made a microphone with the name of the TV station on it, out of a paper cup and a pencil. Another group had booms made from rolled-up paper. One group interviewed "people on the street" - went up to people from other groups and got their impressions. One of the most fun classes we've had so far. It's fun because the time limit is real: I told them the news would air at 9.30 cos it's the 9.30 news, and then started announcing the news even though a couple of groups weren't quite ready. They improvised the rest.
Anyway it worked for us! I'm intrigued by coloured wolves at the disco - I might just try that one!
Katy.
------------------------
From: "Jennifer Wallace" Jennifer0052 (at) hotmail.com
Date: Sat Jun 9, 2001
Subject: Oral Exams
A while back I volunteered details of the oral exam I gave my students last semester. Having recovered from laptop problems, here it is.
This exam was for first year/freshmen college diploma and university degree course students (all english majors). In other words this is the first oral exam they've ever had. This pretty simple test was successful in that the poorest were able to respond appropriately, albeit in one word responses, and asking me almost one word questions, while the higher level students you could barely shut up! I recorded the exams, and did the marking afterwards. I've given my marking scale at the end of all this.
This follows the format used for the speaking test in the first level of Cambridge (UCLES) EFL examination: Key English Test (KET). It uses a very fixed script: this means every student gets the same - no extra help, no forgetting things, etc.
Each student is allowed between 5 and 8 minutes.
Part 1
The examiner interviews the student: the student responds to questions, including one extended response.
Functions covered: giving factual information about yourself, explaining and giving reasons, talking about past experiences
Part 2
The examiner gives a topic for the student to ask questions about (which uses verbal prompts). The examiner answers the questions, as a genuine interlocutor. Functions covered: requesting information (This may include information on: likes / dislikes,habits, factual information)
Part 1
What's your full name?
How do you spell that?
What's today's date?
What time is it? (I HAVE A CLOCK IN THE ROOM)
Where are you from in Anhui?
Did you go to middle school there? (Where did you go to middle school?)
At school, what subjects (ONE OF):
did you like best
were you good at
were the most difficult
didn't you like?
Why?
Any 3 of these questions:
What do you usually do at the weekends?
What did you do last weekend?
What are you going to do this weekend?
What do you do in your free time?
Why do you like ...?
How often do you play / listen / watch ... ?
Have you been to any other towns / places in China?
(Which towns have you visited?)
What did you think of ....... ?
(Did you like ... ?) (What did you / didn’t you like?)
1 of these topics:
Tell me something about your family.
Tell me something about (name of home town).
Tell me something about your hobbies.
Part 2
I'm going to give you a card.
Ask me some questions about ....
Use the words on the card to help you.
Ask me 4 questions.
I'll answer your questions.
Do you understand - shall I repeat that?
FAVOURITE FOOD
What?
How often?
When?
Expensive?
SPARE TIME
What?
How often?
When?
Expensive?
MUSIC
Like?
What type?
Favourite?
When ... listen?
FAVOURITE SPORT
What?
Play?
Watch?
When?
That's the end of your examination - thank you very much.
ASSESSMENT SCALE
Grade
A (100 - 90)
B (89 - 80) Some of the features of A and some of the features of C.
C (79 - 70)
D (69 - 60) Some of the features of C and some of the features of E.
E (59 and below)
-------------------------------------------------------------
From: "eveinbeijing" ear7 (at) hotmail.com
Date: Sat Dec 15, 2001
Subject: oral english assessment rubric
The Lion of Tianjin (HuangDi) sent me this link, and I am posting it with his permission. The link is to a simple rubric which allows for easy, logical grading of oral exams. I plan to use it with my OE exams. I'd like to give it to students in advance so they can know where their grade will come from, but I'd have to explain the fine differences between synonyms like "halting" and "choppy" and simplify teacher-jargon such as "emerging control of basic language structures." LOL...I can just see myself trying to get that concept across, with the blank looks on my students' faces. Maybe I'll just tell them they get from 1-4 points on each of the 5 areas of Task Completion, Comprehensibility, Fluency, Pronunciation, Vocabulary, and Grammar. The rest of the wording of the rubric will be my secret.
http://www.fcps.k12.va.us/DIS/OHSICS/forlang/PALS/rubrics/1spk_an.htm
Eve Ross
Beijing Institute of Machinery
--------------------------------------------------
From: George grose00803 (at) yahoo.com
Date: Wed May 15, 2002
Subject: Re: (teach) Testing Speaking Skills - Middle School
Hi Tony,
To accurately test my students, I give them oral exams which are recorded on tape. These exams have two parts. The first part is Q&A covering things we have covered in class. They almost always have a memorized response for the basic questions. I tend to ignore these. I focus on their responses to the followup questions. For example, I've told them that we might discuss their grandparents, soI might ask
"Are your grandparents alive?" "How many children did they have?" How many boys and how many girls.? "Do you know your aunt's and uncles?" "O.K let's talk about your youngest aunt" Here is where they begin to breakdown because they didn't think to prepare for a discussion about their youngest aunt. I've also begun ba asking about a favorite middle-school teacher and them focus on the teacher they liked the least. Once I gotten to the real subject I'll begin with what is the persons name, age etc. and gradually lead to more complex questions. Then I start looking for syntactic, grammatical and vocabulary failure. In many cases the exam has ended in 2 or 3 minutes and some have gone as long as 30 or 40 minutes. In all cases I use subjects they are familiar with. Family, School, Friends and Hometowns. If I knew more about sports I would dwell on that. I have been know to ask a student to explain what a mid-fielder, a striker or a goalie does if they play those positions in football or the role of Guards, the Center or Forwards in basketball. I've even asked guitar playing students to explain how to play a particular son. In short they give me a guitar lesson.
To test for middle school, determine what is grade appropriate and start from there.
Again, start simple and progress to the complex. At what level do they abandon an answer or the topic entirely. The second part is a short oral reading which incorporates most of the english phonemes. I sometimes give the samples to practice with but they get a new reading for the exam. The must read cold.
Also, I've just begun developing a set of reading passages tha will begin at about fifth or sixth grade level for native speakers using Flesch-Kincaide RGL measures and which become progressively more advanced. This way I can determine the level at which they begin to break down, identified by their rate of word abandonment. In the first year I will be mainly concerned with phonetic identification and production. As we progress, stress and intonation will become more of a factor.
I've not seen the CET tests, so I can't comment on those. If they are as reliable as the written portions of the exam then they should be about as useful as a fireplug at a mama dog's convention. Oral exams can be quantified, but I don't like using them as the basis for a grade. I tell the school that grades shoud be considered as a report of a student's speaking level and how much they have improved. In my classes, the only one's who acutally fail are those who only show up for exams and the rare film. Those who come to class but aren't there count as absent. Our school weeds them out pretty quick. Last term eight of my students flunked out including two who were pretty good english speakers. Six were expelled for cheating on Chinese teacher's exams.
GR
-----------------------------------------
From: "Jennifer Wallace" Jennifer0052 (at) hotmail.com
Date: Wed May 15, 2002
Subject: Testing
Lots of us are trying to develop tests appropriate for the situations we're teaching in. One document I'd recommend, because I've found it enormously helpful, is the Council of Europe Frameowrk, which is on the Internet, as a downloadable pdf file (for which you need to have Adobe Acrobat Reader on your machine). I like the document for several reasons.
The work behind it is the work of a large number of experts across Europe, who've developed one framework to cover the teaching (and testing) of any of the languages taught and used in Europe - which of course includes a variety of non-European languages. In other words, the whole thing is language independent. I understand it to be very much a reflection of the most up to date understanding we have of measuring language performance. The particular document in question is the latest version, the result of many revisions.
The document addresses the fundamental questions in all this, and looks at every dimension conceivable - so I can use it as a basis for testing speaking, listening, reading, anything. It looks at things on general levels and on detailed specific levels - so you can home in on the level that is relevant for you at the moment.
Because this framework is as comprehensive as it is, it's let me think up a variety of activities for the form of my tests, activities that reflect the students experiences and what they've done in a course. But at the same time it's kept me very much on track, enabling me to see clearly what level our target it.
Because it's not language-specific, you can test yourself (there's one section on self-testing) for your Chinese to see how this sort of approach works.
Someone also commented about examiners' ability not to be swayed - well, I think what allows me to be more objective is using a number of scales and criteria when I test. For example, this semester my college end-of-first-year students will get some marks for pronunciation (because we've done quite a bit of pronunciation work on their Oral English classes), some marks for fluency, some marks for grammar, some marks for vocabulary/lexis and some marks for coherence. I'm also thinking about including some marks for how they deal with problems - repair work, asking for help, paraphasing, miming, using fillers to gain thinking time and to fill a silence, and the suchlike - what's called strategic competence. My criteria for vocab/lexis and grammar will not be whether they demonstrate use of anything in particular, but in how effective they are at communicating successfully - do their errors interfere with communication, or hinder it, or render it impossible! This is because I teach college English majors - I think testing for specific aspects of these dimensions is the responsibility of other teachers in other classes. but at the same time, my students do realise that I consider grammar and lexis to be seriously important.
As regards a quick test, my experience, and the experience of other testing large numbers quickly for summer schools (in UK language schools), is that in an informal chat of around 5 minutes, grading only on a 5 pint scale (with very easy to understand scoring 5) is a remarkably effective tool in the hands of a native speaker. Even on the most mundane of topics (your home town, your family), it sorts the lower from the higher from the in betweens. I did this at the beginning of this year with my 225 new students, and on subsequent reflection, having taught them now for 2 semesters, remarkably few of my initial assessments were wrong, and none were way off. What's interesting is looking back at their subsequent development! The value for me is how much respect I have for the students who got a low rating at the beginning who would only now get a middle rating - but wow, what progress! In each band, I can see students who have really made big efforts and made progress, and I can also see students who've made almost no progress. Of those, a small number are not interested in the effort it entails (basketball etc is more important), but I also have one or two who I realise are making efforts but little progress. I think that initial testing and placement has really helped me, and I plan to do it for future Oral English classes.
A while back I asked for help from this group about group work - and got excellent advice, which has really changed our (me and the students) experience of group work. One thing I did was use the test results to make groups according to level, and that's been very successful as well.
Jennifer Wallace
Anhui Gongye Daxue, Ma'anshan
---------------------------------------------------------------------
From: "Fergus Power" ferguspower (at) yahoo.com
Date: Thu May 16, 2002
Subject: Re: (teach) Testing
The Council of Europe framework document can be found at:
http://culture2.coe.int/portfolio/documents_intro/common_framework.html
which is a site headed: A Common European Framework of Reference for Languages
Fergus
---------------------------------------
From: "Alan Simpson" sningbo (at) hotmail.com
Date: Thu Dec 19, 2002
Subject: oral exams
This is our first year teaching oral English. We only keep our students for 1 term, so we need to give exams beginning Jan. I have told my students (freshmen) they need to be able to speak a min of five sentences, & I will give them a choice of topics but only at the time of the exam. Otherwise they will just parrot phrase what they have rehearsed. I think the world of my students, however I have never come across a race that all feel it's o.k. to cheat. Between my wife & I we have 674 students. Any ideas where we can drum up enough topics & any other tips. I have been told it's very rare to fail anybody!
Alan
---------------------------------------------------------------
From: "The Lees" bhic (at) hotkey.net.au
Date: Fri Dec 20, 2002
Subject: Re: (teach) oral exam subjects
http://www.lcc.gatech.edu/regents/samples.html has a lot of debate topics which are probably too advanced for your students. Last semester I found a site which had about 250 simple topics eg what is the best thing about your hometown. What is your favourite colour, what do you look for in a friend, what is your favourite movie, how would you make the world a better place, etc. I put them on a spreadsheet and then numbered them in another column and then used the sort function on random blocks of numbers and topics to mix them all up. There were a very few that were not relevant eg 'Suggest a practical use for Bush's brain' - just kidding -- and these were first deleted.
I cut the sheet up into strips of 6 topics and gave one to each student 10 minutes before their test and they had to choose one out of the six to talk on for about a minute. You could hand them out with a minute to prepare, or whatever. Optional - ask them a question relating to their speech to test impromptu speeaking ability (believe me, 10 minutes notice is enough time for them to write 3 pages and memorise the lot.) Cue cards were ok but reading from notes was not. I tested them in groups of three (in my spare bedroom actually as ther classroom acoustics are terrible) as testing in front of the class is very unfair to the later students as the class will quickly get bored (they are not at all interested in listening to each other).
I found it impossible to be consistent unless I used a marking rubric - simple is better as there is too little time to get too technical. Trying to make comparisons is impossible within a class let alone across different classes.
The other method which would probably have been even fairer would have been to take each student number, multiply it by 465, take the last two digits and allocate that number to that student. Then scale all those scores to lie between 85 and 99 and those are the final marks. Much quicker and as you said "everyone passes"
I will try and find the original list and post the address but it would not take long to enter in a hundred or so topics - hometown, friends, plans for the future, family, occupations and you could have several differnt random sorts of the same topics to keep them guessing. Just make sure they bring in the list and speak on a topic actually on the list. Otherwise they will soon find out roughly what a few of the topics are and prepare a speech on one and hope you don't notice it is not on the list.
TonyL
----------------------------------------------------
From: "SA" satinaa (at) hotmail.com
Date: Mon Jan 6, 2003
Subject: grading oral exams
Now that I'm through my first experience of giving mid term grades for an oral exam, I've got a bunch of questions. You who have been doing this longer, please send me your ideas!
In the States I used a standardized test, so I knew that I was getting an accurate, though relative, measure of the student's oral skills, before and after a term. But that standardized test doesn't work here, because the content is not relavent to what the students here are learning.
For my beginners, this term-end, I modified the standardized test to include the material they are meant to be learning. The test results were generally so low that I couldn't integrate them into the final grade--it would have pulled some students' grades down so low they'd have failed the term. What I was after was a measure of growth from September to now, more than an absolute grade of how they speak now. Since I modified the oral test from September to now, I couldn't use the scores. At the end of spring term, I should be able to apply the same test and get a measure of growth. But how is the best way to incorporate that into the final grade? Use it as one tenth of the final written exam? Include it in classroom participation/attendance/classroom tests portion of the grade?
Is the grading method fairly standardized in China (the formula used for combining finals, classroom participation/attendance, inclass tests, oral skills), or is it an inaccurate science?
For my higher level (English dept.) students I used a very unsatisfactory method. They were happy, all the grades were in the 90s, except those who really couldn't speak, they got in the 80s. I had each student stand before the class and deliver a three minute talk. I suggested a couple of topics, but let them choose anything. I had a score sheet, marking some grammar (s/v agreement, tense), some pronunciation (stress, intonation), structure (introduction/body/conclusion) and overall clarity/comprehensibility. I found it very hard to actually hear and mark all those things, and there were egregious errors I really wanted them marked off for, but hadn't included in my scoresheet.
I gotta do better next term. How do you handle it?
**************************************************
Satina Anziano
Kangding Teachers College
-----------------------------------------------------
From: "dk" davkees (at) public.guangzhou.gd.cn
Date: Wed Jan 8, 2003
Subject: RE: (teach) grading oral exams
Satina Anziano is trying to test the change in speaking ability after a period of a few months of English training.
Satina, the difficulty you experienced is due to the fact that it is not really possible to do that. I was trained as an IELTS examiner for the speaking test. From what I learned from that plus what I learned from further research on the subject I feel this sort of test is too blunt an instrument for such careful comparisions.
Research has shown that an examiner can only measure about 10 levels of English speaking. Trying to measure more leads to inaccuracies. Even to measure over this scale is challenging and examiners need standardization to maintain accuracy.
Other research has indicated that 6-months of English training "can" result in improving by one level. However, even though IELTS is a 'high-stakes' test and students are highly motivated to do well on it, there were also examples of students who did not show a change and even fell one level in test results after the training. This was attributed to possible other factors unrelated to the student's English knowledge but perhaps due to test performance issues.
Really the best way to measure your student's progress is to tell them what you are going to teach them, teach them, tell and review what you taught them and then test them on what you taught.
It's by far easier to test them on what you taught them. They should have absorbed this. You shouldn't test them on things you didn't teach them.
Best wishes,
Dave Kees
Guangzou
-------------------------------------------------------------
From: Nelson Bank natlunla (at) yahoo.com
Date: Tue Jan 7, 2003
Subject: Re: (teach) grading oral exams
Now that I'm through my first experience of giving mid term grades for an oral exam, I've got a bunch of questions. You who have been doing this longer, please send me your ideas!
Hi Tina, My second-language classes in the States' grades were a weighting of daily class grades, bi-weekly tests, and the final exam. I gave a point for a mostly-finished exercise daily (actually 4 or 5 one-pointers), then took bi-weekly tests directly from textbook exercises, and the mid-term and final were announced, explained, and worked out in class, from material from the semester. The big exams were the important things, and we actually did the exam in class several times before "taking" it.
------------------------------------------------
From: "Tony and Betty LEE" leesinchina (at) yahoo.com.au
Date: Tue Jan 7, 2003
Subject: Re: (teach) grading oral exams
I learnt from the speech sessions in normal class not to have exams the same way. The first couple of speakers had the courtesy of the class keeping relatively quiet but those at the end had an impossible time concentrating and the marker an equally impossible time hearing, especially in the echo chambers we used for classrooms.
For my first exam i sent all but three outside - supposedly well out of sight and sound well down the corridor and around the corner. It didn't work because the corridors are even more echoey and they kept drifting closer and those that had finished stayed too close.
All the rest I held in our apartment. A few prepared in the living room and three at a time in the spare bedroom. Perfect.
Experts say they can objectively judge all sorts of faults in a short time and of course, given special training, a good marking rubric, and constant practice they probably can, but the rest of us just have to use a marking scale and try to judge each student by that rather than making comparisons between students -- impossible to avoid entirely but do your best.
A marking scale of 80 to 100 will meet with everyone's approval anyway
TonyL
---------------------------------------------------------
From: Suzannelazicki (at) yahoo.com
Date: Tue Jan 7, 2003
Subject: Re: (teach) grading oral exams
Satina,
Take a look at "Oral Interviews: Activies and Scoring Sheet" at this link:
http://uk.cambridge.org/elt/businessexplorer/resources/activities.htm
Students speak in pairs for my oral examinations, and this lucid scoring sheet nicely covers the areas I wish to grade.
Suzanne
--------------------------------------------------------
From: "Ryan Schreck" schreck517 (at) hotmail.com
Date: Wed Jan 8, 2003
Subject: Re: (teach) grading oral exams
From: Tony and Betty LEE leesinchina (at) yahoo.com.au>
but the rest of us just have to use a marking scale and try to judge each student by that rather than making comparisons between students -- impossible to avoid entirely but do your best.
Are you saying that we shouldn't grade students in comparison to their classmates, or that is too difficult? As I am certainly not an expert in this, I am always concerned by the subjectivity of grading oral exams and I've found through experience that what works best for me is to give them a mark of 1 to 10 (or 1 to 100 if you prefer) and a rank in the class of 1 to 5. Then, I use the ranking to roughly arrange the students according to overall ability and can then use the exam scores, participation and any other factors to determine their final grades. For me, having that initial framework of A students, B students, etc. is incredibly helpful.
Ryan
-------------------------------------------------------- From: Willboots (at) aol.com Date: Wed Jan 8, 2003 Subject: Re: (teach) grading oral exams-methods
Natlunla wrote:
Now that I'm through my first experience of giving mid term grades for an oral exam, I've got a bunch of questions. You who have been doing this longer, please send me your ideas!
Reply:
Dear Tina:
Following is the criteria outline my Univ. provided to all English teachers for the oral exam test. There is a separate 12 subject(s) list the teacher may assign or give the student the option of selecting his/her own subject. I wrote these out on 12 seperate slips of paper and let the student draw from a hat. For instance: one of the topics...."Money is important in our lives. Do you think money can buy happiness?" The assignments were given out the day before the test. We did not allow the student to bring notes to the test. A harder variation is to let the student draw and then speak extemporaneously to the class. This was all but impossible due to the "Face" factor. However, if I was teaching teachers, I would use this method.
The exam should last no longer than 3 to 5 minutes mainly due to the large number of students you must administer in one sitting which was 35 kids for three 50 minute classes in our case. I would confer with my T.A (an accomplished English major) after each test and if I said the student was an "8" and she said a "9" we would agree on an "8.5." As a Chinese she tended to be more liberal than I in grading. She took care of all the grading records in English and Pinyin. Hope this is helpful to some of the list members. Obviously the test is subjective in nature and does not take into account all the good points Dave Kees made in his posting for level of accomplisment during your teaching tenure. Only my T.A. was a true 9.5 but would be a 10 by Chinese standards.
Criteria for Oral English Test
Score (10 points) Grade Description
9-10 Excellent
Natural English with minimal errors and complete understanding of the set task.
8-9 Very good
More than a collection of simple sentences with good vocabulary and structures. Some non basic errors.
7-8 Good
Simple but accurate realization of the set task with sufficient naturalness of English and not many errors.
6-7 Pass
Reasonably correct but awkward and non-communicating or fair and natural treatment of the subject with some serious errors.
5-6 Weak
Original vocabulary and grammar both inadequate to the subject.
0-5 Very poor
Incoherent. Errors show lack of basic knowledge of English.
Notes: Teachers can decide the way of testing. Single work, pair work or group work are all acceptable.
Don in USA
-------------------------------------------------
From: fshdt (at) umac.mo
Date: Thu Jan 9, 2003
Subject: Re: (teach) grading oral exams-methods
Oral grading is not easy and I'm rarely happy with grading systems because to grade you need to break down the student's performance and when you do this it's hard to analyse the many separate elements going on at once. It's easier if you have two markers. I also find it easier to test students in small groups. If you interview each student yourself you become involved in the communication and it's that much harder to make judgements, even though being the interviewer means you call the shots and can set hurdles for the candidate. My wife videos each group so that she can rerun again and again. This is really useful but you do need good sound quality for this so you need good equipment.
I generally mark under 3 categories, on a 1 - 5 scale. The actual scale varies according to my expectations of the students. There's not much point in making 5 'near NS standard' if I'm testing groups of near beginners since this would mean they all score 1. Advanced students would have a more advanced 1. multilevel groups a marked with broader brush strokes. Similarly the nature of the group discussion varies according to the level. Beginners greet each other etc. Advanced learners may discuss academic topics or whatever.
1 Accuracy of language, command of structure, fluency, breadth of vocabulary, complexity
2 Pronunciation, stress and intonation
3 Communicative ability - ability to use language to relate to others in the group, involve others in the group. ability to use coping strategies to avoid linguistic stumbling blocks.
Each of these categories covers a lot of ground but it becomes to complicated to give a mark for every aspect of the student's performance. The item in 3, ability to relate to and involve others in the group is there in part to make sure students co-operate and don't try to compete and elbow out the weaker members in their group. When I explain the grading system to students I emphasise this.
Because each category is so broad the whole thing becomes subjective. It's easy enough to give pronunciation, stress and intonation equal weighting but it's pretty hard to assess fluency anyway and balancing it against breadth of vocabulary etc. can be tricky.
Dick
--------------------------------------------
From: "alan simpson" sningbo (at) hotmail.com
Date: Wed Apr 30, 2003
Subject: oral exams
I was given assistance in organising my oral exams last term from teachers on this list. I teach Freshmen, their English spans from understanding nothing to holding a good conversation.
Because of time constraints & average class sizes of 35 this is what I did last time. I had 108 different topics in a box. The student was asked to draw 5, then from the 5 to pick 1. The student then spoke for 1 minute on that subject. I had many subjects to eliminate cheating. Once a subject was picked, it did not go back into the box. It was a one to one situation.
I stayed away from group discussion as one speaker tends to dominate the conversation. I also made it one to one because of the embarrassment in front of their peers.
I'm now having second thoughts & would like to get your input on the oral English exams which you conduct.
Thanks very much
Alan
------------------------------------------------- From: "Linell Davis" linelldavis (at) hotmail.com Date: Tue Apr 29, 2003 Subject: oral exams
Alan says, "I'm now having second thoughts & would like to get your input on the oral English exams which you conduct."
I no longer teach oral English but when I did I conducted the exams in pairs. I had a smaller number of topics, 8 or 10. The students knew what they all were, but drew their topic from a box at the time of the exam. The pair then have a conversation on the topic. I observe and grade them. I didn't do this in front of the whole class, but arranged a room that students came to at intervals.
Linell
---------------------------------------------------
From: "Tony Lee" leesinchina (at) hotmail.com
Date: Tue Apr 29, 2003
Subject: oral exams
Alan Simpson wrote:
I was given assistance in organising my oral exams last term from teachers on this list. I teach Freshmen, their English spans from understanding nothing to holding a good conversation. Because of time constraints & average class sizes of 35 this is what I did last time.
I had 108 different topics in a box. The student was asked to draw 5, then from the 5 to pick 1.The student then spoke for 1 minute on that subject.I had many subjects to eliminate cheating. Once a subject was picked, it did not go back into the box.
very similar to my system -- pick one of 5 randomly allocated topics (out of 200) -- 5 minutes to prepare and then come in in threes (just so they are not by themselves) and each give their one minute speech. I did have an extra bit where we all discussed something familiar -- eg meal quality on campus -- but that was not helpful in scoring so next time it will be dropped. Will be replaced maybe with one followup question on their chosen topic.
main thing is to have a marking rubric and try to stick with it rather than be tempted to make comparisons.
TonyL
---------------------------------------------
From: fshdt (at) umac.mo
Date: Wed Apr 30, 2003
Subject: Re: oral exams
TonyL wrote:
main thing is to have a marking rubric and try to stick with it rather than be tempted to make comparisons.
TonyL
The marking criteria are the hardest part. They need to cover content, breadth and use of vocabulary, accuracy (grammar and syntax), pronunciation, intonation and stress, turntaking skills (to prevent one or two students monopolising), appropriacy of response, ability to choose suitable register and you could go on.
It's impossible to mark all these so one goes for a general mark or a mark in general areas like accuracy, speech and content, but all the factors need to be there in the background as you weigh up a speaker. We have about 300 students at the same intermediate level, in 15 sections with 5 different teachers. It's vital that we agree standards and monitor each other because however hard you try, oral marking is subjective and we all have to get ourselves to the same place in "oral marking space" and avoid any hint of unfair marking.
Dick
----------------------------------------------------
From: "Suzanne Rowe" aussiesuzanne (at) yahoo.com.au
Date: Wed Apr 30, 2003
Subject: Oral exams and 'th'
Alan wrote in asking about oral exams. Here is one thing I've done in the past.
I give the students a list of topics based on the contents of the course on which to prepare a 2 minute dialogue. They can even memorize their dialogue if they like but of course they can't bring in notes to the exam. I explain to them exactly how the exam will be held and the criteria by which they will be graded.
On the day of the exam, I have the students come in in pairs (as they've organized themselves for their preparation of the dialogue). I make a little small talk with them, and then set them to their dialogue. After they've done what they've prepared, I then ask them some questions based on the topic they've chosen, and then move onto other topics - either related or not, depending on how much I want to stretch them. There ends the exam.
The marking is based on a scale of 0-5 (weakest to strongest) in five different areas - pronunciation, range of vocabulary; fluency; grammar and sentence complexity.
I copied this idea from another foreign teacher and really like it because it gives the students the confidence they need in that they can at least work hard to prepare the memorized part. Also, by having them come in pairs, it cuts down a bit on the length of the exam - a class of 50 takes about 2 hours this way. AND of course, it can be marked fairly easily, although I suppose I should record the conversations so that I can go back and reconsider later.
snip
Suzanne
----------------------------------------------
From: "dk" davkees (at) public.guangzhou.gd.cn
Date: Wed Apr 30, 2003
Subject: Re: (teach) oral exams
Alan says, "I'm now having second thoughts & would like to get your input on the oral English exams which you conduct."
The way you conduct your oral test is the same as I have always seen done here in China.
However, I would like to raise some questions about this method. It was my interest in testing and acquisition that got me into becoming an IELTS examiner and I've been interested in testing for years. These questions may not relate directly to your situation. Perhaps you have already taken measures to mitigate some of these problems raised here but I am taking a broad look at this situation.
It seems most discussion on this point has been related to the physical management of the test concerning the number of students to test at a time and the selection method of speaking topics. I would like to talk about the evaluation method.
What are we testing here? Is this a test of what they learned in your class during your training? Or is this a test of what they know whether they learned it in your class or not? In other words, is this a test of language acquisition or a test of language level or proficiency?
Commonly, the sort of testing oral English teachers do is to give top marks to the students who speak English the best and the ones who speak English the worse the bottom marks. Is this fair?
If a student did not have the language skills you taught during your training and did an excellent job to acquire them during your training what measure will enable that student to get 100% even though he may not be able to speak English as well as the student who knew English well before he attended your training?
How do students feel if they make a great effort to learn the lessons and succeed but cannot get 100%?
If a test is being used which does not seem to be a correct or accurate measure of student effort and progress, to what extent do teachers make up for it by fudging or inflating the grade?
Are there any matters of ethics involved here? Should teachers object to a testing method the school wants them to use that seems arbitrary and potentially damaging to students?
To what extent are teachers tempted to give a lower test score to students who were disruptive during the training?
I think that all of these are important aspects that have to be considered and accounted for as we go about our testing duties.
Regards,
Dave Kees
------------------------------------------------
From: "Nancy Buswell" nancybuswell (at) yahoo.com
Date: Thu May 1, 2003
Subject: Re: Oral Exams
Dave Kees had some good points in his message. I've wondered myself in my five years of teaching in China what the best way to grade is. It seems unfair to grade on the basis of their speaking level, which may or may not have anything to do with what they have learned in your class. I think it was Leslie who wrote a year ago that the best way would be to test them at the beginning and then at the end to see how much progress was made. I don't have the skills to do this, and don't want to take the time. My solution is to basically grade them on their interest and effort. They show their interest by attending class and participating. They show their effort by doing assignments. I haven't perfected (ha!) my method yet, but my idea for the final exam is from a high school teacher I had long ago who told us to make a contract to decide what grade we will get, based on how much work we wanted to do. To apply that to my university oral English class, I told the students that if they were happy with an 85 (assuming that they had participated in class, not had any unexcused absences and had done all the assignments), then they wouldn't have to do anything for their final exam. If they wanted to make a 90 or higher, they could get together with a friend or two and create a 3-minute short play on a subject that I give them. They have one week to work on the play. While they practice with their friends they are getting English practice in, and that's what we want, isn't it? To refine my idea, I might have two levels of final exam assignments, one harder than the other, but I haven't tried this yet. Note: I'm not sure that this idea will work with English majors, who are graded more strictly. I have only used it with non-English majors.
Another note: Feel free to lambast my method at will. Many of you have more training and education than I do.
Nancy, who got a job offer in Xi'an yesterday (they sounded desparate)
--------------------------------
From: "dk" davkees (at) public.guangzhou.gd.cn
Date: Thu May 1, 2003
Subject: Re: (teach) Re: Oral Exams
"I think it was Leslie who wrote a year ago that the best way would be to test them at the beginning and then at the end to see how much progress was made."
I. ~~~
This is a very logical and obvious approach to the problem. Again, we have to define the What of testing. Proficiency testing can be much more difficult than you might guess for several reasons.
1. According to research, it is too difficult for highly trained examiners to measure English proficiency spanning more than 9-10 levels. (Covering a scale of no ability to highly proficient.)
2. Even measuring 10 levels professional examiners can be wrong 27% of the time.
3. On such a scale, for classes that meet once or twice a week for 45-minutes, it may take a full year of training or more to improve one level
4. Factors weigh on the whole process which can cause inaccuracies. During some research on IELTS training it was found that after a 3-month intensive training candidates could improve half a level. However, some candidates actually scored at a lower level at the end of the training than they did at the beginning. Reasons for this were the state-of-mind of the candidate on the test day, familiarity and lack of familiarity with the subject matter of the tests, faults in the testing system (IELTS), etc. So it would be possible to test your student at the beginning of the course and then the student could do worse at the test at the end of the course. How would you give a grade in this case?
II. ~~~
Alternatively, and what may be a better solution albeit too late at this point, is to test the students on the course material at the beginning of the course. If the students are properly placed in the right level classes they should score very low on such a test but after training on the material should score very high at the end of the course.
In that it is too late at this point, the next best thing is to review the material we taught the students and find some distinct and important points that we can test on. However, if the classes consisted of "well, what does everybody want to talk about today?" it may be impossible to test the students.
III. ~~~
Finally, the idea of scoring by student attitude or participation in the class or other student behavior focused method, to my mind, but perhaps many would disagree with me, is fraught with the most disadvantages. Such a criteria makes the test exactly one of classroom behavior, not learning or language acquisition. As the focus of most of us teachers is on the student centered classroom we are0 constantly questioning ourselves if we are meeting the needs and capturing the interests of our students. Some days we do better than others. Some students (especially the very bright and the very dull) are more easily bored than others. We are tempted to punish our troublemakers and reward our so-called 'good' students by the score we would give them. But again, we would be scoring the student on their behavior which is perhaps not the best way to reflect their learning in our class.
~~~~~~~~~
The irony is that for many schools it really doesn't matter what score you give the students because the school often does not treat the oral English class taught by a foreigner as a 'real' class. These grades often don't show up as part of their year-end scores.
Nonetheless, I think we should all be careful in how we go about making these kinds of decisions. There are certain 'automatic' impulses that we should be aware of and question. How often do we do things because that's the way everyone else does them? How often do we do things because that's the way it was done when we were students?
These things I have said in the above are with the realization that they may not exactly apply to the certain aspects of the current discussion. Without complete understanding of the situation I know I may be misunderstanding some things. I am just trying to explore various factors and considerations for the purpose of reflection.
We are the teachers and in a position of power with the students and the school. It is a great opportunity to for us to explore and discover the best ways to do things for our student and ourselves.
Regards,
Dave Kees
Guangzhou
--------------------------------------------
From: fshdt (at) umac.mo
Date: Thu May 1, 2003
Subject: (teach) Re: Oral Exams
In my position I can't grade by improvement rather than proficiency because the students eventually get sent out into the world with a piece of paper saying they have obtained a degree taught in the medium of English. If they came to the university with practically no English and improved to, say, lower intermediate, it's not enough. If they leave with a degree and poor English, employers are not going to trust our qualifications.
I agree with Dave, the whole thing is subjective and depends a lot on the state of the student on a particular day, perhaps more so than other exams testing writing or reading. It's just that we have to do something and so we struggle to do the best to be fair and reduce subjectivity. Having two markers helps.
Dick
------------------------------------------
From: "Tony Lee" leesinchina (at) hotmail.com
Date: Fri May 2, 2003
Subject: Re: (teach) oral exams
"dk" davkees@p... wrote:
I would like to talk about the evaluation method.
What are we testing here? Is this a test of what they learned in your class during your training? Or is this a test of what they know whether they learned it in your class or not? In other words, is this a test of language acquisition or a test of language level or proficiency?
snip
If we use the mathematical method of 'reductio ad absurdum' we must ask if the following is justified.
proposed that - if a raw beginner comes into university and goes out as an intermediate learner do you award a PhD on the basis that the improvement was infinite and if a perfect speaker comes into university and goes out a perfect speaker, do you award a fail because there was zero improvement.
If the answer is no, then all degrees of the proposal are also false.
Another way of looking at it is that it is easy to make big progress when you start from a low base but very difficult at higher levels -- i.e. law of diminishing returns
TonyL
-------------------------------------------------
From: Anne Quaine ankuini (at) yahoo.com.au
Date: Fri May 2, 2003
Subject: Re: (teach) oral exams
In response to several others, Tony Lee leesinchina (at) hotmail.com wrote:" ... if a raw beginner comes into university and goes out as an intermediate learner do you award a PhD on the basis that the improvement was infinite and if a perfect speaker comes into university and goes out a perfect speaker, do you award a fail because there was zero improvement ..."
I quite agree with Tony's point about the relative standards and the PhD, but unlike a recognised academic award, does a letter grade for oral English have any real meaning to any of the parties involved, including prospective employers, if there is no agreed standard that it is supposed to represent? It would seem to me that how and what I tested in 'oral English' would depend on who was paying me, and to do what. If an employer asked me to teach oral English, I would want to know what their expectations were - how they saw my role. Is it simply a side issue to the main academic study - to increase the students' confidence, fluency and pronunciation etc in speaking (and listening skills) as I see fit? Or is there a target set of language and skills that are taught to? Then I would want to know exactly what they wanted assessed and how it fitted into the overall course assessment. ie, does it 'count'? Or is it merely a guide for students to be ranked and for the auditors to know that something was being done?I suspect there is no one-size-fits-all response to the question. No doubt a descriptive assessment of where students are currently at would be the fairest and most meaningful, but this is not how things are usually done here, is it?
Anne
---------------------
From: fshdt (at) umac.mo
Date: Sat May 3, 2003
Subject: Re: (teach) oral exams
I quite agree with Tony's point about the relative standards and the PhD, but unlike a recognised academic award, does a letter grade for oral English have any real meaning to any of the parties involved, including prospective employers, if there is no agreed standard that it is supposed to represent?
It doesn't have any meaning so long as there are no standards for grading. Even so, if you give an A grade to someone who can't even tie their shoelaces in English an employer will not be happy with your grading.
No doubt a descriptive assessment of where students are currently at would be the fairest and most meaningful, but this is not how things are usually done here, is it?
Anne
Yes, profiling would be fine, and no, it's not how things are done here. We need to work for change so that we can do a decent job. "Change does not come from above." The Internationale (Billy Bragg's version)
Dick
-----------------------------------------------------------------
From: "Tony Lee" leesinchina (at) hotmail.com
Date: Sat May 3, 2003
Subject: Re: (teach) oral exams
Anne Quaine ankuini@y... wrote:
...but unlike a recognised academic award, does a letter grade for oral English have any real meaning to any of the parties involved, including prospective employers, if there is no agreed standard that it is supposed to represent?It would seem to me that how and what I tested in 'oral English' would depend on who was paying me, and to do what. ... Anne
University is a rather special situation and China is no different to USA where students proudly give their grade averages on their resumes. The number of scholarship awards the Chinese student gets is a good indication of his exam passing ability and employers use it as part of their assessment.
One of the students at my last university received a scholarship of 6,000rmb. If his mark on any subject -- including oral english -- had been 1% lower, his award would have been only 500Yuan. This is why in some places the students are desperate to get at least 75% because if they get below that in any subject they can't get a scholarship no matter how high their aother marks are. Interestingly it is the politics subject that most of them come unstuck on.
TonyL
-----------------------------------------------
From: "alan simpson" sningbo (at) hotmail.com
Date: Mon May 5, 2003
Subject: Oral Exam
Thanks to all who gave input about arranging oral exams.
We are not from a teaching background & welcome the input provided by professionals. However, without slinging off about anybody personally we are surprised about the lack of professionalism displayed by some teachers both where we teach & others we have spoken to.
We have an open mind to suggestions, but strongly feel that there should be uniformity. We have come across :- Teachers who are prima donnas, and who will not enter into discussion about their teaching; Teachers who have been in the profession for many many years, yet make the most basic mistakes, i.e.always going down the [attendance] registrar to ask questions so the same students are picked each week & the ones at the end don't get asked; Giving half the class an exam & advising them afterwards that was the exam, then advising the other half their exam is next week & telling them what the topics will be; A teacher who claims Chinese should be taught a sense of social responsibility, where if they clean his apartment, they can get extra marks; Teaching oral English & only the teacher speaks; The comment that you test the competence at the beginning & then mark as to the progress made. If this was the case & Einstein was in the class but didn't make any effort, he would get a lower mark than a student that is poor but made progress.Surely it doesn't matter where they got the knowledge & it's ability that counts. I have employed many people and if they have certificates, I expect them to be able to do the work ; Being unprepared & running out of material halfway through a lesson.
I repeat we're not professionals, and the following is how we think we will conduct our oral exam.
We are employed to teach & test on Oral English only. This does NOT include testing for social obligations, writing & spelling, this is for the Chinese English teachers to decide. We are also not here for testing public speaking, many native speakers would be nervous to speak in front of an audience. We feel we should provide the best possible atmosphere possible to ensure the student is not nervous. We also feel there cannot be any favouritism, such as testing by simply going down the attendance register as nobody wants to be first. Names should be picked randomly.
We will test one to one, give 30 secs to think, 2 minutes to talk. Subjects will not be advised in advance & will be enough to ensure there cannot be cheating. Marking will be based on the flow of speaking, number of sentences & construction, vocabulary & grammar. If the student freezes or runs out we will prompt but not ask questions. A note will be taken of number of the prompts given. Half of the class will be selected randomly to do the exam one week, followed by the next half the following week. We are undecided as to whether we will do this in the classroom or apartment. Failures would be re-tested in our own time, in case the student was having a bad day. Our exam will constitute 60% of the overall mark, the 40% balance will be the mid term test, home work, participation & attendance.
Trust we have not opened up a can of worms. We invite your detailed comments on the above.
Alan
-------------------------------------------------------
From: "Pete Marchetto" p_ottehcram (at) hotmail.com
Date: Mon May 5, 2003
Subject: Re: Oral exams
Tony L:
... if a raw beginner comes into university and goes out as an intermediate learner do you award a PhD on the basis that the improvement was infinite and if a perfect speaker comes into university and goes out a perfect speaker, do you award a fail because there was zero improvement.
Wo:
I'm with Tony on this one... but with one notable exception in my first year here, a student who started at a very low level, (a rural student), with spectacularly low confidence. One of my proudest achievements as a teacher was finding myself able to boost his confidence - albeit somewhat by accident - to such a degree that he went from being totally closed to a happy chatter in the course of a single lesson. (Long story).
Come the examination I confess to boosting his position in the rankings by a couple of places more than was warranted, this sudden surge coming too late in the year to be of much value in terms of improvement, but this being his first year with plenty of room for further improvement to come. In part I was trying to compensate for other classes in which I knew he would be poorly marked and, still fearful he might be penalised by the department in some way for low scores, wrote a letter of recommendation to the department encouraging them to give him a chance, explaining the situation and my confidence that he would show a marked improvement in his second year.
I don't make a habit of this sort of thing, but I think there can be room under certain circumstances for a little 'creative marking' when it comes to the particular circumstance of the student. This might particularly apply early on in a student's studies when the student has started at a low level and you have reason to believe that, in coming years, the efforts of that particular student will result in a marked rise compared to his or her classmates. Similarly, a privileged student who makes little effort to improve upon an already high standard might be marked down somewhat by way of a kick up the backside.
In final year examinations I wouldn't dream of doing this; those should simply reflect a student's ability. However, I believe that in a student's first year of study - to a lesser extent his or her second - there may be exceptional circumstances in which a mark may be skewed to reflect a student's attitude and comparative performance, thus more closely reflecting perhaps the student's potential and the probable outcome of his or her studies as a whole.
Pete Marchetto
------------------------------------------------
From: distant_todd (at) yahoo.com
Date: Wed May 7, 2003
Subject: Re: Oral Exam
Soon I will have to conduct exams for the students in my oral English classes (English majors). I understand that in my department these normally take the form of interviews. I haven't decided on the content yet, but I want to test the candidates' spontaneous speech so perhaps I will provide a few general topics beforehand (so that they know what to expect), and then hit them with very specific questions in the interview.
As for evaluation, if it was any other kind of class I would only test them on material taught during the semester (ie if it was a geometry class I wouldn't give them calculus questions), but since the aim of the class is to improve their spoken English there are no specific language points to test. Even if we did happen to focus on "politely disagreeing" in week 3, it's a bit hard to elicit this language in a short interview! So proficiency testing is the only choice. Possible criteria are: pronunciation, intonation, fluency, grammatical accuracy, range of vocabulary, clarity of structure. Indeed, if the university provides a score sheet it will probably look something like this. But it's a bit unrealistic to imagine I can judge all these aspects accurately in a few minutes.
The marking guide I will really follow is a suggestion I once saw in a book: 5 marks for fluency; 5 marks for accuracy (including pronunciation).
I said earlier that the aim of the class was to improve their spoken English. That's why I think it's vitally important that attendance and participation make up part of the final mark. We have to assume that students who speak during class are improving their fluency! Note that by "participation" I do not mean volunteering answers, but participating in group activities (ie not reading a book or speaking Chinese during the tasks). To motivate the students, the details of the final marking scheme should be told to them in the first week. (I have to admit that I still haven't finalised it, and thus haven't told them a darn thing! Some details I've only started considering after reading this thread, which has been quite enlightening.) It would also be good to tell the students the breakdown of their marks after the exam, so they can identify their weaknesses.
Any comments on these ideas?
Todd
------------------------------------------------
Note: Exams are a perpetual topic -- bring them up anytime on the TEFLChina email list (teflchina@yahoogroups.com). Mention the URL of this page. Thanks :-) Roger