英语教育测量与评价.doc

研究生学位课程试卷

院（系、所） 外国语学院 专业 英语
考试科目 英语教育测量与评价 第 二 学期
研究生姓名 戎竞雄 学号 132300176

考试成绩
导师评语：

导师签字
年 月 日

说 明
一、凡学位课程考试试题、试卷必须与本封面一起装订。阅卷导师务必用红笔批卷，并在本封面规定位置打分、写完评语后在二周（论文考试一个月）内交院（系、所）办公室教务员，教务员及时做好成绩登记，在学期结束前或第二学期初将成绩单交研究生处统一整理归档。试题、试卷由院（系、所）办公室保管。
二、学位课程考试用纸除计算机专用打字纸、16开小方格稿子纸外，一律使用研究生处统一印制的“学位课程考试纸”。
三、该封面请用A4纸双面打印，将此说明打印于封面背面。
上海师范大学标准试卷
2013 ~ 2014 学年 第二学期 考试日期 2014 年8 月 日
科目：英语教育测量与评价
学科教学＋课程与教学论 专业 硕士 13 年级 姓名 戎竞雄 学号 _132300176__
项目
一
二
三
四
五
六
七
八
总分
分值
45
55

100
得分

我承诺，遵守《上海师范大学考场规则》，诚信考试。签名：__戎竞雄_____
I. Answer in detail the following questions: ( 45%, 15 points for each)
1. Suggest the differences between proficiency tests and achievement tests. Give examples if necessary.
Answer: A proficiency test assesses the general knowledge or skills commonly required for entry into a group of similar institutions. One example is TOEFL. Proficiency tests are norm-referenced tests because NRTs have all the qualities desirable for proficiency decisions. While an achievement test must be designed with very specific reference to a particular course. And the achievement tests are often directly based on course objectives and will therefore be criterion-referenced. Such tests will typically be administered at the end of a course to determine how effectively students have mastered the instructional objectives. Achievement tests must be not only very specifically designed to measure the objectives of a given course but also flexible enough to help teachers readily respond to what they learn from the test about the students' abilities, the students' needs, and the students' learning of the course objectives. One example is the tests at the end of the course.

2. The following are two different kinds of score distribution.
.

What information do these two figures convey us? (Discuss from the score distribution of NRT and that of CRT)
Answer: The first figure conveys us that it is a norm-referenced test, which is designed to measure global language abilities (for instance, overall English language proficiency including listening ability, reading comprehension, and so on). Each student's score on such a test is interpreted relative to the scores of all other students who took the test. Such comparisons are usually done with reference to the concept of the normal distribution (familiarly known as the bell curve). The purpose of an NRT is to spread students out along a continuum of scores so that those with poor language abilities are at one end of the normal distribution, while those with "high" abilities are at the other end (with the bulk of the students falling near the middle).
The second figure shows it is a criterion-referenced test (CRT), which is usually produced to measure well-defined and fairly specific objectives. Often these objectives are specific to a particular course or program. Each student's score is meaningful without reference to the other students' scores. A student's score on a particular objective indicates the percent of the knowledge or skill in that objective that the student has learned.
Moreover, the distribution of scores on a CRT need not necessarily be normal. If all the students know 100% of the material on all the objectives, then all the students should receive the same score.
The purpose of a CRT is to measure the amount of learning that a student has accomplished on each objective. In most cases, the students would know in advance what types of questions, tasks, and content to expect for each objective.

3. What is reliability and validity? What is the relationship between reliability and validity? To assess a candidate’s oral language ability in an oral test, the examining body often asks two examiners to score that candidate’s performance. Similarly, when an examiner is grading a composition for a certain test, i.e. TEM4, the same composition can be marked by the same examiner on two occasions. Explain in detail why such measures should be taken.
Answer: The test reliability is defined as the extent to which the results can be considered consistent or stable. Test validity is defined here as the degree to which a test measures what it claims, or purports, to be measuring.
If teachers administer a placement test to their students on one occasion, they would like the scores to be very much the same if they were to administer the same test again. The degree to which a test is consistent, or reliable can be estimated by calculating a reliability coefficient, which can go as high as +1.0 for a perfectly reliable test or as low as 0 when the results on the test are totally unreliable. Once the tests are administered twice and the pairs of scores for each student are lined up, simply calculate a Pearson product-moment correlation coefficient between the two sets of scores. The correlation coefficient will provide a conservative estimate (that is underestimate) of the reliability of the test over time. This reliability estimate can be interpreted as the percent of reliable variance on the test.
Test validity is defined here as the degree to which a test measures what it claims, or purports, to be measuring. For example, if a test claims to measure proficiency in German listening comprehension, that is just what it should assess.

II. Discussion (55%, 17 points for 1 and 2, 21 points for 3)
1. Look at the following table and answer the questions that follow:
1) Calculate the total standard scores for the two students
2) Compare the total standard scores between the two students, see which student scored higher, and explain briefly why a teacher had better use the total standard scores instead of the total raw scores.
Subject
Mean
SD
Student A
Student B
Psychology
81
6
85
80
Writing
85
9
80
91
Listening Comprehension
70
5
76
85
Reading
74
10
93
66
Literature
88
3
90
95
Total

424
417
Answer :
1)
Subject
Mean
SD
Student A
Standard
scores
Student B
Standard scores
Psychology
81
6
85
0.667
80
-0.17
Writing
85
9
80
-0.556
91
0.667
Listening Comprehension
70
5
76
1.2
85
3
Reading
74
10
93
1.9
66
-0.8
Literature
88
3
90
0.667
95
2.33
Total

424
3.88
417
5.03

2)从上图所计算出的标准分可以看出虽然学生A 的总分比学生B的总分高，但
学生B的考试得分其中三项的标准分显然比学生A 要高，也就是大部分的标准分比学生A 要高。从标准分总分来看，学生A的标准分总分为3.88，学生B的标准分总分为5.03，高于学生A。所以学生B考得好些。不同考试是不同质的，把不同考试的分数求和是没有意义的，同时也难以真实反映学生的整体情况。所以正确的做法是先将各门课程的考分转换成标准分，再求和，标准分求和后的总分加以比较，从而判断学生A，B在考试中的优劣，这样才更科学。

2. Use SPSS 16.0
(1) to calculate the correlation coefficient between two sets of writing scores marked by two teachers for the same group of students and then make the scatter plot. Discuss if there is any correlation between these two sets of scores.
(2) to do the dependent sample T test to see whether there is any significant difference between the two teachers’ marking of the same paper. Report and discuss the result. If there exists significant difference, give suggestions as to how to solve this problem.
(3) to calculate the correlation coefficient between three sets of writing scores marked by three teachers for the same group of students and make the scatter plot between Rater 1 and Rater 2; Rater 1 and Rater 3, and Rater 2 and Rater 3. Discuss if there is any correlation between these three sets of scores.
(4) to do ANOVA to see whether there is any significant difference among the three teachers’ marking of the same paper (use LSD and SNK, and draw Means Plot). Report and discuss the result.
（1）

相关性

A组教师
B组教师
A组教师
Pearson 相关性
1
.321
显著性（双侧）

.073
N
32
32
B组教师
Pearson 相关性
.321
1
显著性（双侧）
.073

N
32
32

从以上的相关性显著性值0.073,大于0.05，以及散点图，可以看出两位教师所批改的写作分数无相关。
（2）

成对样本检验

成对差分
t
df
Sig.(双侧)
均值
标准差
均值的标准误
差分的 95% 置信区间
下限
上限
对 1
A组教师 - B组教师
-3.250
8.316
1.470
-6.248
-.252
-2.211
31
.035
从上表成对样本检验中的Sig.(双侧)值0.035，小于0.05，可以看出两位老师所打的分，有明显差异。可以通过算标准分来解决此问题。
（3）
相关性

Rater1
Rater2
Rater3
Rater1
Pearson 相关性
1
.632**
.599**
显著性（双侧）

.000
.000
N
55
55
55
Rater2
Pearson 相关性
.632**
1
.652**
显著性（双侧）
.000

.000
N
55
55
55
Rater3
Pearson 相关性
.599**
.652**
1
显著性（双侧）
.000
.000

N
55
55
55
**. 在 .01 水平（双侧）上显著相关。

从以上的相关值和散点图可以看出，Rater 1 and Rater 2的相关度不高；Rater 1 and Rater 3属于正相关；Rater 2 and Rater 3属于较强的正相关。

（4）

分数

组别
N
alpha = 0.05 的子集

1
Student-Newman-Keulsa
3
55
68.95
1
55
69.40
2
55
71.75
显著性

.302
将显示同类子集中的组均值。
a. 将使用调和均值样本大小 = 55.000。

从以上表格中alpha的值为0.302，大于0.005，可以看出三个数值没有显著差异，也就是三位老师的打分没有明显差异。

3. A teacher wants to apply a new teaching method in one of the two classes he is teaching and to see whether this method can be effective. Before he begins his experiment, he uses the T-Test of SPSS to compare the scores of the mid-term examination between the two classes and he finds the significance value is 0.547. Then, he adopts the new method in EC (experiment class) and for CC (comparison class), he still uses the old method. After the two months of experiment, the teacher uses the T-Test of SPSS again to compare the scores of the final examination between the two classes and this time, he finds the significance value is 0.03. Discuss the importance of the two significance values from two T-Test of SPSS.
答：从这位老师对实验班和对照班所做的期中考试的T检验的显著性水平值为0.547，大于0.05，可以看出前侧两个班无差异；两个月后期末考试的T检验的显著性水平值为0.03，小于0.05，可以看出后侧两个班有差异，由此，可以看出这位教师所采用的新方法对学生是有效的。