A study on the reliability of the final achievement computer-Based mcqs test 1 for the 4th semester non - English majors at hanoi university of business and technology

est was relevant to what they had learned. The result is shown in the chart below: Chart 1: Students’ response on test content Among four sections, functional language was perceived as the most relevant with the total proportion of 65%. Reading section was claimed to be the least relevant (31% only). Vocabulary was said to be a little more relevant to the syllabus than grammar (59% compared with 52%). Giving opinions on the test length, three fourths of the students (75%) found that the total number of 150 multi choice items was reasonable for them. 25% of them thought that it was too many. Answering the question whether the test as a whole had power to discriminate among students in the ability of interest, approximately 36 % of students determined that test items actually discriminate the student level of English. The rest of 64% claimed the level of discrimination was not remarkable. The result can be seen clearly in the following pie chart: Discrimination value 36% 64% high low Chart 2: Students’ response on item discrimination value In the fourth question, students were asked if they had enough time to fulfill the tasks given in the achievement test1. The following chart illustrates the result: Time length 84% 9% 7% enough not enough too much Chart 3: Students’ response on time length By observing the result in Chart 3, we realize that roughly 84% of students answered time management was not a problem for them. 7% of responses showed that time allowance was too much while 9% said that they needed more time to finish the tasks. Regarding the clarify of the test instruction, 90 % of student stated that the instruction was clear. Only 10% of them perceived it was quite unclear. When being asked about the influence of test supervision on the test result, 98% of students commented that test supervisors were strict. Only 2% of them acknowledged that they were under no very strict supervision. Students were also asked whether testing room affected their performance. 40 % of them claimed that the testing room did have impact on their test performance. 60% stated they were not affected. Responding to the question whether they experienced computer breakdown when doing the test and whether their test results were affected, a third of informants stated that they did and had to do the test again. 77% of them found it have a very negative influence on their test performance. The rest of 23% saw no impact. When being asked if they suffered from physical and emotional pressure when performing tasks, 45% of students admitted they did while 55% of them did not. With reference to test-taking behavior, 56% the informants responded that they did select the answers arbitrarily whereas 44% did not. The result was illustrated in the chart below. : Response Arbitrariness 56% 44% Yes No Chart 4: Students’ response arbitrariness Answering the question about prior exposure to the test format and content, 97 % of students realized that they were familiar with this type of test. And only 3% were not. This can be explained that they were the second year students and have done a number of tests. Concerning students’ computer skill, 61% of students claimed that they were good at using computer to do the test. 38% thought their skill was normal. Only 1% stated that they were not good at it. When asked whether any difference between doing the test with hard copy and soft copy exists, amazingly 50 % of the participants found it different and 50% did not although they were the second year student and experienced four times doing MCQs English test on computers. In the last question, students were asked whether the test scores reflected their actual achievement during the 4th semester. The result was presented in the following pie chart: test score and students' achievement 66% 34% exactly not exactly Chart 5: Students’ response on relation between test score and their achievement As it can be seen from Chart 5, 66% of students acknowledged that the test score actually reflected their achievement while 34% of them did not get the score as expected. From these results we can realize some points as follows: - Factors which do not affect students’ scores include students’ computer skill, students’ familiarity with the test format and content, test supervision, clarity of test instruction, and time allowance. - Factors affecting students’ test performance involve test characteristics, testee characteristics and test administration characteristic. Test characteristics include a large number of test items, low content relevance to the course book and low discrimination power. Testee characteristics consist of response arbitrariness, suffering from pressure and bad ability of reading texts on the screen. Test administration characteristic involves computer breakdown. Clearly when performing tasks, students were heavily influenced by both objective and subjective factors and therefore the results they got did not reflect their true ability as 34% of them claimed. In short, the test scores do not seem reliable from students’ perspectives. That is because students’ performances on the test were affected by a number of both objective and subjective factors. All of the findings to three research questions mentioned above lead to a conclusion that the MCQs test 1 does not yeild a reliable result. The unreliability of the test resulted from the performance of both test-takers and test-designers. As for test designers, they made the test of low quality. The allocation of items with difficulty among four sections was not reasonable. The items were also not really discriminating. As for test-takers, they did not perform the tasks well. Notably, according to the findings obtained from the comparison and analysis of test item content, there is high relevance between the test and the course book, especially in reading section. However, the findings from the questionnaire survey for students show that the test content is not actually relevant to what students have been taught, especially reading part. It is likely that the flunctuation from students when doing the test such as pressure, difficulty in reading texts on computers and response arbitrariness made them believe that the content of the test was generally 50% relevant to what they have learnt and their test scores does not reflect their true ability. Regarding to all aspects, the MCQs test 1 has one good point. That is, it is valid in terms of content. Nevertheless, this point is not enough to conclude that it is a good test as it lacks reliability. 5.4. Pedagogical implications and suggestions on improvements of the existing final achievement computer-based MCQs test 1 for the non-English majors at HUBT. In this section, some suggestions for test-designers are offered to improve the quality of the final achievement MCQs test 1. A good achievement test must be valid and reliable. In order to make a more valid achievement test, test designers should stick to the course objectives of developing speaking and listening skill when designing achievement tests. According to Table 4-section 3.2.2. illustrating time allocation and skill weighing, speaking and listening skill are the main focuses of the course book Market Leader Pre-intermediate, these skills therefore should be tested with relevant skill weight proportion. Furthermore, functional language section in the MCQs test should be removed as it is far from the real – life situations. In fact, appropriate responses to various stimuli in everyday situations should be produced rather than chosen from these limited responses. Instead, functional language should be included in speaking tests. The scoring format for semester 4 should be as follows: Semester 4 (12 credits) The first score (6 credits) The second score (6 credits ) Oral test 25% Oral test 25% Paper test 1 - listening - writing 35% 25% 10% Paper test 1 - listening - writing 35% 25% 10% Computer-based MCQs test 1 - reading - vocabulary - grammar 45% 20% 10% 15% Computer-based MCQs test 1 - reading - vocabulary - grammar 45% 20% 10% 15% Table 14: Suggested scoring format It is expected that this suggested scoring format should ensure the principle of “test what is taught”. In order to improve the test reliability, it is necessary to establish a testing committee of three to five people who will be responsible for test construction, administration and development instead of only one as it is at present. The testing committee should be made up of members with good knowledge, skills and experience of making MCQs tests. They are recommended to pay attention to the following three issues. First, the testing committee members on one hand should design MCQs tests themselves and on the other hand require teachers to make their own tests. Teachers should be provided with test design and test development techniques involving vocabulary, grammar and reading by the testing members so that they can construct tests of good quality. This can be done through regularly-held workshops. The main reason for this is no one can understand the students’ strengths and weaknesses better than these teachers. Therefore the tests made by them can be sure to be reliable and practical with the students. Both committee members and teachers need to clarify students’ levels of language in order to maximize the test efficiency. This information would be helpful for them to avoid designing items with undesirable difficulty and discriminality value. In addition, the content of the test should relate to and familiar to what the students are taught and learnt during the course as much as possible. The test should also be systematically built up on the ground of a carefully constructed test specification. Second, these test items should be carefully taken into consideration regarding the relevance to the course book content and then only acceptable tests item should be selected and piloted to students. The trial can be done at classrooms with strict supervision and it is preferable to enable students to do the test on computers in order to help them to get familiar with reading soft copy texts and to reduce their pressure. Third, the results obtained from the trials should be carefully analyzed and discussed in terms of test difficulty, test discrimination, instructions, time allowance, distractors in order to decide which items are good enough and which items need adjusting to put into an item bank. The item bank therefore can guarantee the variety of test choices, test quality and test confidentiality. Last but not least, the item bank needs to be updated, supplemented and adapted, especially after the achievement tests are given to students in each semester, with items of good quality for the consolidation and development of a standardised one. * A proposed Specification Grid for the final achievement computer-based test 1 for the 4th semester non –English majors in HUBT. Based on the findings of the study and the course objectives, a proposed test specification of the current 4th term English achievement MCQs test 1 is worked out as follows so that more accurate measures of students’ language competence can be achieved. The objectives of the final achievement objective test 1 for the 4th term non – English include: Checking what the students have learnt about vocabulary, grammar and reading and to what degree the objectives of the course have been achieved in the set timeframe. Assessing students’ achievement at the end of the course, especially to evaluate students’ results after learning three units of Market Leader pre-Intermediate book. Giving students’ feedback. The test results will be useful for students to see what they have achieved in their learning process. Identifying room for improvement for both teaching content and teaching methodology. That is, teachers will refer to their students’ scores/ errors to adapt their teaching methods, the syllabus content and materials so as to make them more appropriate to their students’ needs and abilities. The following is the grid of this tests’ specification. Achievement test: Paper specifications grids Time allowance: 150 minutes Level: Pre – intermediate for non- English majors (Hanoi University of Business and Technology) Test of Reading, Grammar and Vocabulary Section Main skill focus Input Response/ item type Number of marks Skill weighting 1. Reading Reading for gist/ specific information including topics closely related to marketing, planning and managing Narrative or factual text. Approx. 60-80 words each X 41, 4 multi choice option 4.1 41% 2. Grammar Recognizing grammar items involving wh-questions, future expression and reported speech Narrative or factual text, approx.15-20 words each X 32, 4 multi choice option 3.2 32% 3.Vocabulary Recognizing vocabulary items including noun-noun , verb- noun and verb-preposition collocation Narrative or factual text, approx. 15- 25 words each X 27, 4 multi choice option 2.7 27% Table 15: Proposed test specifications 5. 5. Summary In this chapter, results and conclusions about three research questions of the study are drawn out and discussed. The findings show that the final achievement computer-based MCQs test 1 for the second year non-English majors at HUBT is to a certain extent not reliable. Thus some suggestions to make the test more reliable and high quality are given to test designers. Chapter 6: Conclusion 6.1. Summary of the findings Test reliability is undeniably an important criterion to define the quality of a test. The investigation and evaluation of the reliability of the final achievement computer-based MCQs test 1 are therefore useful to the judgment of the quality of teaching, learning and testing process at HUBT. Through data collected from students’ test score and item responses, the author find out the answers to three research questions about the compatibility of the test objective, test content and test format to the course objective and the syllabus content, the extent to which the test scores are reliable and the students’ attitude towards the test and then to come to a final conclusion about the reliability of the test. The findings of indicate that the MCQs test 1 is not a good test as it first of all lacks compatibility between the test objective and the course objectives. The skill weight format of the test and of the syllabus are also incompatible. Four sections of the MCQs test 1 cover language items in the course book but the coverage relevance is still problematic. In addition, the MCQs test 1 fails to meet one of the most important criteria- reliability . The unreliability exists due to some problems. First, test items are of low quality as a result of low item difficulty and item discrimination value. Item analysis and students’ perception of the test discrimination indicate that the test does not have good discrimination value. Students’ perception and reliability coefficient of the MCQs test 1 both also show that the test score that students gets are unreliable. Second, several characteristics involving test items, testees and test administration such as a large number of test items, low content relevance to the course book, response arbitrariness, pressure and ability of reading text on the screen and computer breakdown as perceived by students reduce the reliability of the test scores. On the basis of these results, the author provides some suggestions towards the improvement of the test quality. The reliability of the final achievement MCQs test 1 for second year non-English majors may be increased if it is constructed more relevant to the course objectives and syllabus and if test items are designed and withdrawn from an item bank of good p-value and discrimination value by an efficient testing committee. The author hopes that the study will give a detailed view of the Computer-based MCQs tests administrated at HUBT and the suggestions towards the test improvement will come into reality in order to properly assess students’ actual language ability during the process of learning Market Leader Pre-intermediate. 6.2. Limitations of the study The study on the reliability of the final achievement Computer-based MCQs test does contain some unavoidable limitations. Firstly, this thesis investigated only a minor aspect among a lot of facets of test reliability due to the limit of time and the scope of a minor MA thesis. That is internal consistency reliability. Secondly, the test item analysis does not include distractor tally which can bring a much deeper view on the test due to the fact that access to these data was impossible. Finally, the author only developed a set of questionnaire to evaluate the test reliability from students’ perspective. If the attitude and perception of the teachers on the test had been studied, the results would have been more comprehensive. 6.3. Suggestions for further study Considering the important of testing and the existence–to-a-certain-degree of the unreliability of the computer-based MCQs test, further research is needed to study its effects on language learning and assessing and identify coping strategies to help students promote their learning of four English skills while MCQs is still employed as a very useful testing technique. References 1. Alderson, J.C., Clapham, C. and D. Wall. (1995). Language Test Construction and Evaluation. Cambridge University Press. 2. Bachman, L. F (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press 3. Bachman, L.F; Palmer, A.S (1996). Language testing in practice: designing and developing useful language tests. Oxford: Oxford University Press 4. Brown, H. D. (1995). Teaching by principles. An Interactive Approach to language Pedagogy. London: Longman 5. Cotton, D. , David , F. and K. Simon . (2002). Market Leader- Pre-intermediate. Longman. 6. Harrison, A. (1983a). A Language Testing Handbook. London: McMillan Press 7. Henton, J.B. (1988). Writing English Language Test. Longman Group U. K. 8. Henton, J.B. (1990). Classroom testing. New York: Longman 9. Henning, G. (1987). A guide to Language Testing: Development, Evaluation, Research. Cambridge: Newbury House Publishers 10. Hien, T.T. (2005). The pros and cons of the multiple-choice testing technique with reference to methodological innovation as perceived by secondary English language teachers and students. Unpublished M.A Thesis, VNU. 11. Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press. 12. Kunnan, A.J; Milanovic, M. (2000). Fairness and validation in language assessment : selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida . Cambridge : Cambridge University Press, 13. Lynch, B.K (2003). Language assessment and programme evaluation. Edinburgh: Edinburgh University Press. 14. McCOUBRIE, P. (2004). Improving the fairness of Multi choice questions : a literature review. Medical Teacher, Vol 26, No. 8, 2004, pp709-712. 15. Mc Namara, T. (2000). Language Testing. Oxford: Oxford University Press 16. Milanovic, M. (1999). Issues in computer-adaptive testing of reading proficiency. Cambridge : Cambridge University Press 17. Milanovic, M; Saville, N. (1996). Performance testing, cognition and assessment : selected papers from the 15th language testing research colloquium (LTRC), Cambridge and Arnhem. Cambridge : Cambridge University Press 18. Spolsky, B. (1995). Measured words. Oxford: Oxford University Press 19. Trang, H.V. (2005). Evaluating the reliability of the achievement writing test for the first-year students in the English Department, College of Foreign language, Vietnam National University, Hanoi and some suggestions for changes. Unpublished M.A Thesis, VNU5. 6. 20, 20. Weir, C. J. (1990). Communicative Language testing. Prentice hall International (UK) Ltd. 21. Weir, C.J. (2005). Language testing and validation: an evidence- based approach. Basingstoke : Palgrave Macmillan. APPENDICES APPENDIX 1 Grammar, Reading, Vocabulary and Functional language check list Part Unit Items Task Page no Grammar 7 Questions Correcting/ Making questions/ Completing and Matching questions P66; 29-30 8 Talking about future plan Matching/ completing sentences/Making sentences/ P74; 32-33 9 Reported speech Completing sentences/ Transforming direct into indirect speech/ Building sentences P82; 37 Reading 7 Selling dreams – Beyond advertising- Fun marketing Answering question/ Ordering headings/ Matching/ True-False/ Classifying P65; 126-127 8 Planning for tourism – Time management- Town planning Matching/ Answering questions/ Numbering summary/ Completing sentence/ Answering Qs P72-73; 128-129 9 Managing across cultures- The company song- Coaching new employees Answering the questions/ Matching / True-False/ Choosing correct alternatives P81; 130-131 Vocabulary 7 Word partnerships Filling missing vowels/ Matching/ Doing puzzle/ Completing a text P. 63; 28,29 8 Ways to plan Matching/ Combining word/ Completing a text P71; 32 9 Verbs and preposition Matching/ Completing table/ Completing sentence/ Making sentence/ Correcting P30; 36 Functional language 7 Telephoning: Exchanging information (checking information, asking for information, finishing a conversation) Listen and tick, answer questions, complete chart/ Role play P 67 8 Meeting: Interrupting and clarifying Listen and order, identify language function/ Role play P 75 9 Socializing and entertaining (making excuses, asking and giving opinion, saying goodbye, thanking hosts) Answer questions/ Listen and answer Qs, complete extract, order/ Role play P 83 APPENDIX 2 C©u hái ®iÒu tra C¸c em sinh viªn K11 th©n mÕn, Khoa TiÕng Anh mong nhËn ®îc ý kiÕn cña c¸c em vÒ bµi thi tr¾c nghiÖm cuèi kú 4 (A6). C¸c c©u tr¶ lêi thËn träng, chÝnh x¸c vµ ®Çy ®ñ cña c¸c em cho nh÷ng c©u hái díi ®©y sÏ gióp Ých rÊt nhiÒu cho viÖc n©ng cao chÊt lîng bµi thi cho sinh viªn n¨m thø hai . Xin ch©n thµnh c¶m ¬n sù céng t¸c cña c¸c em! C¸c em h·y ®¸nh dÊu vµo nh÷ng c©u tr¶ lêi mµ c¸c em lùa chän, vµ ®a thªm ý kiÕn nÕu cÇn thiÕt. 1. C¸c em h·y nhËn xÐt néi dung cña bµi thi tr¾c nghiÖm trªn m¸y. Néi dung cña bµi thi cã phï hîp víi c¸c kiÕn thøc mµ c¸c em ®îc häc trªn líp kh«ng? * PhÇn tõ vùng a. Phï hîp b. Kh«ng phï hîp ………………………………………………………………………………………… * PhÇn ng÷ ph¸p a. Phï hîp b. Kh«ng phï hîp ………………………………………………………………………………………… * PhÇn ®äc hiÓu a. Phï hîp b. Kh«ng phï hîp ………………………………………………………………………………………… * PhÇn t×nh huèng a. Phï hîp b. Kh«ng phï hîp ………………………………………………………………………………………… 2. C¸c em cã nhËn xÐt g× vÒ sè lîng 150 c©u hái trong bµi thi tr¾c nghiÖm cuèi kú? a. Qu¸ nhiÒu b.Võaph¶i ………………………………………………………………………………………… 3. C¸c em cã nhËn xÐt g× vÒ møc ®é ph©n lo¹i tr×nh ®é häc sinh cña bµi thi tr¾c nghiÖm? a. Cao b. ThÊp ………………………………………………………………………………………… 4. C¸c em h·y nhËn xÐt vÒ thêi gian lµm bµi thi tr¾c nghiÖm trªn m¸y. Thêi gian lµm bµi: a. Thõa b. §ñ c. ThiÕu ……………………………………………………………………………………… C¸c em cã nhËn xÐt g× vÒ c¸c chØ dÉn dÉn lµm bµi trong bµi thi tr¾c nghiÖm trªn m¸y ? a. Râ rµng b. Kh«ng râ rµng ………………………………………………………………………………………… 6. C¸c em ®¸nh gi¸ th¸i ®é cña c¸n bé coi thi nh thÕ nµo? a. Nghiªm tóc b. ThiÕu nghiªm tóc ………………………………………………………………………………………… 7. Theo c¸c em phßng thi cã ¶nh hëng ®Õn kÕt qu¶ lµm bµi cña c¸c em kh«ng ? a. Cã ¶nh hëng b. Kh«ng ¶nh hëng ………………………………………………………………………………………… 8. Trong qu¸ tr×nh lµm bµi m¸y tÝnh cña em cã bÞ trôc trÆc kh«ng? NÕu cã vµ c¸c em ph¶i lµm l¹i tõ ®Çu th× viÖc nµy cã g©y ¶nh hëng tiªu cùc ®Õn kÕt qu¶ bµi thi cña c¸c em? a. Cã ¶nh hëng b. Kh«ng ¶nh hëng ………………………………………………………………………………………… 9. C¸c em cã chÞu ¸p lùc vÒ t©m sinh lý khi c¸c em lµm bµi kh«ng? a. Cã b. Kh«ng ……………………………………………………………………………………… 10. Trong khi lµm bµi thi c¸c em cã thêng tuú tiÖn chän ®¸p ¸n kh«ng? a. Cã b. Kh«ng ………………………………………………………………………………………… 11. C¸c em nhËn xÐt g× vÒ kiÓu bµi thi tr¾c nghiÖm trªn m¸y ? a. Quen thuéc b. Kh«ng quen thuéc ………………………………………………………………………………………… 12. C¸c em nhËn thÊy kü n¨ng sö dông m¸y tÝnh khi lµm bµi thi tr¾c nghiÖm cña m×nh nh thÕ nµo? a. Tèt b. Kh«ng tèt ………………………………………………………………………………………… 13. C¸c em nhËn thÊy viÖc ®äc vµ lµm bµi thi tr¾c nghiÖm trªn m¸y so víi viÖc ®äc vµ lµm bµi thi tr¾c nghiÖm trªn giÊy cho kÕt qu¶ nh thÕ nµo? a. Kh¸c nhau b. Kh«ng kh¸c nhau ………………………………………………………………………………………… Theo c¸c em ®iÓm bµi thi tr¾c nghiÖm ph¶n ¸nh sù tiÕn bé trong qu¸ tr×nh häc tËp cña c¸c em ë trªn líp nh thÕ nµo ? a. ChÝnh x¸c b. Kh«ng chÝnh x¸c ……………………………………………………………………………………….… APPENDIX 3 STUDENTS’ TEST SCORES STUDENTS' TEST SCORES EXAMINEE x x (x-x)2 EXAMINEE x x (x-x)2 Score Mean Score Mean 1 Lª Minh §øc 6.13 6.59 0.21 50 Chu ThÞ Ph¬ng 7.6 6.59 1.01 2 T¹ TuÊn Anh 5.73 6.59 0.75 51 §µo Duy Phong 5.33 6.59 1.6 3 Vò TrÇn ChÝnh 7.4 6.59 0.65 52 Ng« V¨n Qu©n 5 6.59 2.54 4 Ng« Th¸I Dòng 7.07 6.59 0.23 53 Ng« ThÞ Th×n 5.67 6.59 0.85 5 NguyÔn ThÞ Hång H¹nh 5.07 6.59 2.32 54 TrÇn ThÞ Th¶o 5.4 6.59 1.42 6 L¬ng Hång H¹nh 4.27 6.59 5.4 55 L¹i V¨n Thêng 6.67 6.59 0.01 7 Ph¹m V¨n Kú 4.93 6.59 2.77 56 N«ng Ph¬ng Thuú 6.87 6.59 0.08 8 §ç Ngäc LuyÖn 6.27 6.59 0.1 57 Ph¹m Ngäc Tó 6.47 6.59 0.02 9 §Æng Xu©n Nam 6.47 6.59 0.02 58 NguyÔn ThÞ Trang 8.73 6.59 4.57 10 Hoµng Quèc Th¸i 6.2 6.59 0.15 59 NguyÔn ThÞ Thu Trang 4.13 6.59 6.07 11 TrÇn Trung Thµnh 5.67 6.59 0.85 60 Lª ThÞ YÕn 6.07 6.59 0.27 12 Phan ChiÔn Th¾ng 6.07 6.59 0.27 61 Ph¹m ThÞ DiÖp 6.47 6.59 0.02 13 Lª B¸ Thùc 5.67 6.59 0.85 62 T¹ ThÞ Doan 6.33 6.59 0.07 14 NguyÔn Quèc Toµn 6.13 6.59 0.21 63 §Æng V¨n Dòng 5.47 6.59 1.26 15 Ng« M¹nh TuÊn 5.27 6.59 1.75 64 Lª Thuú Dung 6.47 6.59 0.02 16 NguyÔn Anh TuÊn 6.13 6.59 0.21 65 Ph¹m ThÞ Duyªn 7.2 6.59 0.37 17 Bïi TrÝ TuÖ 5.13 6.59 2.14 66 §oµn Ngäc H¶i 5.33 6.59 1.6 18 NguyÔn Kim Anh 6.67 6.59 0.01 67 NguyÔn Thi H¹nh 7.33 6.59 0.54 19 Bïi TuÊn Anh 5.93 6.59 0.44 68 NguyÔn V¨n HiÕu 3.4 6.59 10.2 20 Mai Trung HiÕu 5.47 6.59 1.26 69 §inh V¨n Hoµng 7.53 6.59 0.88 21 NguyÔn Xu©n Linh 6.13 6.59 0.21 70 NguyÔn ThÞ Kim Liªn 8.87 6.59 5.18 22 §Æng Ngäc Long 7.6 6.59 1.01 71 Th©n ThÞ Ngäc Mai 5.87 6.59 0.52 23 §ç TiÕn M¹nh 6.13 6.59 0.21 72 TrÇn Hoµi Nam 4.93 6.59 2.77 24 Hoµng Quèc Minh 7.67 6.59 1.16 73 Bïi ThÞ Ng¸t 6.87 6.59 0.08 25 Ph¹m V¨n Phóc 6.53 6.59 0 74 NguyÔn Quúnh Nga 7.6 6.59 1.01 26 Viªn Lª Quang 8.4 6.59 3.26 75 Bïi Thuý Nga 7.33 6.59 0.54 27 Vò V¨n Th¸i 6.47 6.59 0.02 76 §ç ThÞ BÝch Ngäc 6.07 6.59 0.27 28 TrÇn V¨n ThuËn 5.8 6.59 0.63 77 NguyÔn Jen Ny 6.33 6.59 0.07 29 Lª M¹nh Tó 7.33 6.59 0.54 78 NguyÔn Xu©n Quúnh 6.07 6.59 0.27 30 §ång Sü To¶n 5.87 6.59 0.52 79 Vò ThÞ T©m 6.87 6.59 0.08 31 Ph¹m V¨n TuÊn 6.13 6.59 0.21 80 TrÇn ThÞ Th¬ng 8 6.59 1.98 32 NguyÔn ThÞ Tó Uyªn 6.53 6.59 0 81 NguyÔn Kim Thu 5 6.59 2.54 33 Ng« B¸ V¨n 5.47 6.59 1.26 82 Lª Minh Thuû 5.73 6.59 0.75 34 Bïi H¶i Vinh 4.8 6.59 3.22 83 NguyÔn Ph¬ng Thuý 7 6.59 0.17 35 NguyÔn Träng Vinh 8 6.59 1.98 84 NguyÔn ThÞ TiÖp 7.93 6.59 1.79 36 NguyÔn V¨n ViÖt 5.87 6.59 0.52 85 NguyÔn ThÞ Thu Trang 8.13 6.59 2.36 37 §Æng ThÞ Quúnh Anh 7.27 6.59 0.46 86 §Ëu ThÞ HuyÒn Trang 4.27 6.59 5.4 38 Lª Quang B×nh 4.47 6.59 4.51 87 Phan ThÞ Quúnh Trang 6.67 6.59 0.01 39 Chu V¨n ChuyÓn 3.93 6.59 7.09 88 NguyÔn ThÞ H¶i YÕn 6.87 6.59 0.08 40 NguyÔn Ngäc DiÖp 7.53 6.59 0.88 89 NguyÔn ThÞ §am 7.47 6.59 0.77 41 §inh Thu H»ng 6 6.59 0.35 90 §µo Quúnh Anh 8.67 6.59 4.31 42 NguyÔn Ng©n Hµ 6.73 6.59 0.02 91 NguyÔn ThÞ Dung 5.67 6.59 0.85 43 Cao V¨n H¶i 6.13 6.59 0.21 92 Hoµng ThÞ Thu Giang 7.6 6.59 1.01 44 Do·n ThÞ H¹nh 5.4 6.59 1.42 93 Bïi Trêng Giang 6.33 6.59 0.07 45 Bïi ThÞ HiÒn 5.47 6.59 1.26 94 Phan Ngäc H¬ng 8 6.59 1.98 46 Phan ThÞ Mü H¬ng 8.53 6.59 3.75 95 Bïi ThÞ H¶I Hµ 5.93 6.59 0.44 47 §ç ThÞ Linh 6.07 6.59 0.27 96 T¨ng ThÞ Kim H¹nh 5.53 6.59 1.13 48 Vò ThÞ Mai 6.6 6.59 0 97 TrÇn ThÞ Hoµi 7.4 6.59 0.65 49 Lý ThÞ Ph¬ng Ng©n 5.8 6.59 0.63 98 NguyÔn Ph¬ng Hoa 6.87 6.59 0.08 STUDENTS' TEST SCORES EXAMINEE x x (x-x)2 EXAMINEE x x (x-x)2 Score Mean Score Mean 99 NguyÔn ThÞ Thanh HuyÒn 7.07 6.59 0.23 147 Tõ ThÞ Hµ V©n 6.87 6.59 0.08 100 Lu Thuú Linh 6.67 6.59 0.01 148 D¬ng ThÞ H¶I YÕn 7.27 6.59 0.46 101 Ph¹m Thanh Long 6.53 6.59 0 149 Bïi Minh §øc 5.33 6.59 1.6 102 NguyÔn ThÞ Quúnh Mai 9.33 6.59 7.49 150 NguyÔn Quúnh Anh 7.53 6.59 0.88 103 NguyÔn ThÞ Nga 6.93 6.59 0.11 151 Lª ThÞ Quúnh Anh 8.27 6.59 2.81 104 Vò ThÞ Ngäc 6.67 6.59 0.01 152 Tr¬ng Thuú Chi 7.6 6.59 1.01 105 NguyÔn Th¶o Nguyªn 6.93 6.59 0.11 153 Hµ Kim Dung 8.27 6.59 2.81 106 NguyÔn ThÞ Minh NguyÖt 8.73 6.59 4.57 154 PhÝ ThÞ H»ng 6.33 6.59 0.07 107 Th¸i Ngäc Nhung 6.07 6.59 0.27 155 §µo Minh Hµ 8.47 6.59 3.52 108 NguyÔn ThÞ Lan Ph¬ng 6.67 6.59 0.01 156 Ph¹m thÞ Thu Hµ 8.07 6.59 2.18 109 NguyÔn Hå Quanhg 6.67 6.59 0.01 157 Lª ThÞ Hång 5.2 6.59 1.94 110 §ç ThÞ Nh Quúnh 6.33 6.59 0.07 158 Vò ThÞ TuyÕt Lan 5.27 6.59 1.75 111 NguyÔn ThÞ Th¶o 7.4 6.59 0.65 159 NguyÔn ThÞ Liªn 7.53 6.59 0.88 112 Ph¹m ThÞ Ph¬ng Thuú 7.07 6.59 0.23 160 NguyÔn C«ng Nhí 2.93 6.59 13.4 113 Lª ThÞ Thu Thuû 7.93 6.59 1.79 161 Bïi ThÞ Ph¬ng 7.27 6.59 0.46 114 Lª ThÞ Thuû 8.4 6.59 3.26 162 Bïi Mai Ph¬ng 8.53 6.59 3.75 115 Ph¹m ThÞ Ngäc Trang 7.67 6.59 1.16 163 NguyÔn Thanh Phîng 7.6 6.59 1.01 116 NguyÔn ThÞ Thu Trang 6.13 6.59 0.21 164 Bïi ThÞ Thu Quúnh 7.87 6.59 1.63 117 Lª Quúnh Trang 8.67 6.59 4.31 165 Lu ThÞ Trang Th¶o 0.13 6.59 41.8 118 Ng« Quèc Tu©n 5.73 6.59 0.75 166 NguyÔn ThÞ Th¾m 7.33 6.59 0.54 119 Vò ThÞ Ngäc Anh 8.33 6.59 3.02 167 TrÇn ThÞ Thoa 8.2 6.59 2.58 120 NguyÔn Thµnh C«ng 7.07 6.59 0.23 168 Hoµng ThÞ Thu Thuû 7.33 6.59 0.54 121 Bïi Kh¾c Cêng 2.47 6.59 17 169 NguyÔn ThÞ Thu Trang 3.93 6.59 7.09 122 Ph¹m V¨n Dòng 3.6 6.59 8.96 170 Hoµng ThÞ HuyÒn Trang 7.4 6.59 0.65 123 §oµn ThÞ Kim Dung 6.53 6.59 0 171 §Æng ThÞ HuyÒn Trang 8.33 6.59 3.02 124 NguyÔn T Duy 6.13 6.59 0.21 172 D¬ng HuyÒn Trang 5.47 6.59 1.26 125 Hµ Lan H¬ng 5.27 6.59 1.75 173 Ph¹m Thuý V©n 7.13 6.59 0.29 126 NguyÔn ThÞ Thu HiÒn 6.53 6.59 0 174 Vò Thanh Xu©n 9 6.59 5.79 127 Hoµng Thi HiÒn 5.47 6.59 1.26 175 T¹ Quèc §¹t 7 6.59 0.17 128 §Æng Thanh HuyÒn 5.4 6.59 1.42 176 Vò Träng §am 7.13 6.59 0.29 129 NguyÔn Ngäc Linh 6.87 6.59 0.08 177 TrÇn Vò §é 8.07 6.59 2.18 130 Vò KiÒu Loan 7.73 6.59 1.29 178 L· M¹nh Cêng 6.53 6.59 0 131 Ph¹m ThÞ Quúnh Mai 8.67 6.59 4.31 179 TrÞnh V¨n Cêng 5.67 6.59 0.85 132 Vò ThÞ Minh 6.4 6.59 0.04 180 L¹i V¨n Dòng 4.53 6.59 4.26 133 NguyÔn ThÞ Ng©n 7.33 6.59 0.54 181 TrÇn Mü H»ng 5.73 6.59 0.75 134 §ç ThÞ Ng©n 6.47 6.59 0.02 182 NguyÔn Anh Hµo 8.07 6.59 2.18 135 NguyÔn Hoµng Nga 5.4 6.59 1.42 183 Mai Thanh H¶i 6.93 6.59 0.11 136 NguyÔn Hång Nhung 7.27 6.59 0.46 184 TrÇn DiÖu Hång 6.67 6.59 0.01 137 Ph¹m ThÞ Hång Nhung 4.53 6.59 4.26 185 TrÇn ThÞ BÝch HËu 6.4 6.59 0.04 138 NguyÔn TrÇn Ph¬ng 7.47 6.59 0.77 186 Vò ThÞ Hoµng Lan 4.33 6.59 5.12 139 Bïi M¹nh Qu©n 7.47 6.59 0.77 187 §Æng Vò LËp 6 6.59 0.35 140 M¹c ThÞ Ngäc Quúnh 8.67 6.59 4.31 188 TrÇn Thµnh Long 4.07 6.59 6.37 141 NguyÔn Duy S¬n 4.07 6.59 6.37 189 Lu ThÞ KiÒu Oanh 5.67 6.59 0.85 142 NguyÔn Quang ThiÖn 6.07 6.59 0.27 190 NguyÔn V¨n Qu¶ng 5.6 6.59 0.99 143 Ph¹m ThÞ Kim Thu 6.73 6.59 0.02 191 NguyÔn NguyÖt Quúnh 7.6 6.59 1.01 144 §µo Thi Thuý 7.27 6.59 0.46 192 Bïi ViÖt Th¸i 6.47 6.59 0.02 145 NguyÔn ThÞ Thu Trang 4.47 6.59 4.51 193 Hoµng Ph¬ng Th¶o 7.13 6.59 0.29 146 Tr¬ngThÞ H¶i V©n 5.6 6.59 0.99 194 §ç Duy Th¾ng 6.67 6.59 0.01 STUDENTS' TEST SCORES EXAMINEE x x (x-x)2 EXAMINEE x x (x-x)2 Score Mean Score Mean 195 Ng« V¨n Th¸ng 5.87 6.59 0.52 244 Cao Thu Nga 8.87 6.59 5.18 196 Lª Quang Thä 7.2 6.59 0.37 245 Ng« H»ng Nga 6.4 6.59 0.04 197 NguyÔn Xu©n Toµn 6.67 6.59 0.01 246 NguyÔn ThÞ Quúnh 7.13 6.59 0.29 198 Cao Anh Trung 6.33 6.59 0.07 247 NguyÔn ThÞ Minh T©m 7.8 6.59 1.46 199 §ç Ngäc TuyÒn 7.07 6.59 0.23 248 Hoµng ThÞ Th¶o 5.73 6.59 0.75 200 §inh §øc Anh 8.67 6.59 4.31 249 §ç §øc ThiÖn 8.53 6.59 3.75 201 TrÇn TuÊn Anh 8 6.59 1.98 250 §oµn Thu Thuû 5.2 6.59 1.94 202 Ph¹m Ngäc B¸u 6.93 6.59 0.11 251 Hoµng Thanh Tïng 6.67 6.59 0.01 203 Vò ThÞ Dung 7.8 6.59 1.46 252 Lª Quang §¹t 7.53 6.59 0.88 204 TrÇn V¨n Hµ 5.27 6.59 1.75 253 NguyÔn Duy §iÒn 6.4 6.59 0.04 205 §µo ThiÞ Thu Hµ 6.73 6.59 0.02 254 NguyÔn V¨n ChÝnh 9.07 6.59 6.13 206 Ph¹m Trung HiÔu 7.8 6.59 1.46 255 D¬ng Thu H¬ng 8.93 6.59 5.46 207 NguyÔn Quang Huy 7.93 6.59 1.79 256 Vò ThÞ Thuý Hµ 7.87 6.59 1.63 208 Vò V¨n Huyªn 7.27 6.59 0.46 257 §ç Thu Hµ 7.93 6.59 1.79 209 Vò §øc Lîng 6.93 6.59 0.11 258 Vò thi Thu Hµ 7.47 6.59 0.77 210 NguyÔn V¨n LiÔu 7.73 6.59 1.29 259 Ph¹m ThÞ Thu Hoµi 7.2 6.59 0.37 211 §inh TiÕn Lùc 4.8 6.59 3.22 260 Vò ThÞ Ho¹t 8.4 6.59 3.26 212 NguyÔn §øc Minh 6.47 6.59 0.02 261 NguyÔn ThÞ My HuyÒn 6.13 6.59 0.21 213 Tèng Quang Nam 3.2 6.59 11.5 262 NguyÔn Minh HuyÒn 8.27 6.59 2.81 214 NguyÔn BÝch Ngäc 6.4 6.59 0.04 263 TrÇn Mü Linh 6.8 6.59 0.04 215 Bïi Minh Ngäc 8.73 6.59 4.57 264 Bïi Huy Long 6.73 6.59 0.02 216 NguyÔn Thanh T©m 7.33 6.59 0.54 265 Hoµng Long 8.67 6.59 4.31 217 TrÇn M¹nh Th¾ng 8.33 6.59 3.02 266 Ng« Ngäc Mai 8.67 6.59 4.31 218 NguyÔn ThÞ Thanh Thuû 8.2 6.59 2.58 267 NguyÔn BÝch Phîng 7.53 6.59 0.88 219 NguyÔn ThÞ Thu Thuý 6.87 6.59 0.08 268 NguyÔn Hoµng S¬n 3.4 6.59 10.2 220 Ph¹m V¨n D Tïng 6.2 6.59 0.15 269 NguyÔn Hoµng S¬n 5.33 6.59 1.6 221 Ph¹m ThÞ Thu Trang 6.33 6.59 0.07 270 NguyÔn ThÞ T¸m 3 6.59 12.9 222 TrÇn ThÞ Thu Trang 6.33 6.59 0.07 271 NguyÔn ThÕ T×nh 2.53 6.59 16.5 223 §ç Quèc Trinh 6.4 6.59 0.04 272 NguyÔn M¹nh Tëng 3.73 6.59 8.2 224 TrÇn NguyÖt ¸nh 8 6.59 1.98 273 Vu¬ng ThÞ Thu Trang 5.07 6.59 2.32 225 NguyÔn Hång ¢n 7.33 6.59 0.54 274 Bïi Quang Trung 4.8 6.59 3.22 226 NguyÔn Ph¬ng Anh 8.53 6.59 3.75 275 §ç §øc ViÖt 4.67 6.59 3.7 227 Chu ViÖt Cêng 6.53 6.59 0 276 TrÇn Thu Anh 3.73 6.59 8.2 228 NguyÔn ThÞ Minh Ch©u 7.13 6.59 0.29 277 Hoµng Thä C«ng 5.27 6.59 1.75 229 NguyÔn Ph¬ng Dung 8.13 6.59 2.36 278 Cao §øc Cêng 3.53 6.59 9.38 230 NguyÔn Thuú Dung 6.8 6.59 0.04 279 NguyÔn M¹nh Cêng 4.27 6.59 5.4 231 NguyÔn ThÞ Thuý H»ng 6.87 6.59 0.08 280 L¬ng Minh Ch©u 6.73 6.59 0.02 232 NguyÔn Thu H¬ng 7.67 6.59 1.16 281 Vò TiÕn Dòng 2.87 6.59 13.9 233 NguyÔn ThÞ Hång Hµ 7 6.59 0.17 282 Ph¹m Ngäc Duy 4.33 6.59 5.12 234 Lª ThÞ Ngäc Hµ 7.93 6.59 1.79 283 Vò ThÞ H¬ng Giang 6.6 6.59 0 235 NguyÔn DiÖu Hµ 7.2 6.59 0.37 284 Vò Trêng Giang 6.13 6.59 0.21 236 NguyÔn Thanh H¶i 9.4 6.59 7.88 285 NguyÔn ThÞ H¬ng 8.73 6.59 4.57 237 TrÇn Lª Huy 3.07 6.59 12.4 286 Hoµng ThÞ Mai H¬ng 2.67 6.59 15.4 238 NguyÔn DiÖu Linh 8.8 6.59 4.87 287 §inh ThÞ Thu Hµ 8.6 6.59 4.03 239 Lª ThÞ Thanh Loan 8.07 6.59 2.18 288 TrÞnh Thu Hång 7.8 6.59 1.46 240 NguyÔn H¶i Long 4.87 6.59 2.97 289 §ç M¹nh Hïng 7.4 6.59 0.65 241 NguyÔn ThÞ Hµ Ly 6.73 6.59 0.02 290 Bïi SÜ HiÕu 7.47 6.59 0.77 242 NguyÔn ThÕ MÉn 4.2 6.59 5.73 291 Bïi Huy Hoµng 4.53 6.59 4.26 243 Ph¹m thÞ Thuý Ngµ 7.13 6.59 0.29 292 Lª ThÞ Hoa 4.8 6.59 3.22 STUDENTS' TEST SCORES EXAMINEE x x (x-x)2 EXAMINEE x x (x-x)2 Score Mean Score Mean 293 TriÖu Kh¸nh Hoµ 8.87 6.59 5.18 342 §ç Hu¬ng Thuû 8.13 6.59 2.36 294 Vò §×nh Khoa 5.87 6.59 0.52 343 NguyÔn Anh Tó 4.8 6.59 3.22 295 Ph¹m ChÝ Lîng 7 6.59 0.17 344 Ng« V¨n Toµn 9.27 6.59 7.16 296 NguyÔn ThÞ Thuú Linh 6 6.59 0.35 345 TrÇn Hu¬ng Trang 7.2 6.59 0.37 297 TrÞnh ThÞ Minh Loan 6.47 6.59 0.02 346 Phã §øc Trung 4.4 6.59 4.81 298 Vò §øc Long 8.07 6.59 2.18 347 Phan ThÞ V©n 8.2 6.59 2.58 299 NguyÔn TuyÕt Mai 8.27 6.59 2.81 348 Ph¹m Thanh V©n 6.93 6.59 0.11 300 Hoµng ThÞ Kim Oanh 8.13 6.59 2.36 349 Hoµng ViÖt 9.47 6.59 8.27 301 NguyÔn Ph¬ng Th¶o 7.13 6.59 0.29 302 Giang ThÞ Th¶o 8.53 6.59 3.75 ∑x 2301 303 NguyÔn V¨n Thao 6.33 6.59 0.07 ∑(x-x)2 740 304 §inh Ph¬ng Ngäc Anh 3.47 6.59 9.76 Mean 6.59 305 NguyÔn thÞ V©n Anh 6.67 6.59 0.01 306 Ph¹m V¨n B×nh 4.4 6.59 4.81 307 TrÇn ThÞ thuú D¬ng 5.8 6.59 0.63 308 L¹i ViÖt Dòng 7.67 6.59 1.16 309 Phïng ThÞ Ngäc Dung 7.4 6.59 0.65 310 NguyÔn H¬ng Giang 6.93 6.59 0.11 311 Ng« §øc H¶i 7.87 6.59 1.63 312 TrÇn Huy Hng 6.67 6.59 0.01 313 KhuÊt ThÞ Thu Hoµi 6.07 6.59 0.27 314 NguyÔn Hoµi Linh 6.47 6.59 0.02 315 Lª ThÞ DiÖu Linh 8.2 6.59 2.58 316 Ph¹m ThÞ Loan 5.13 6.59 2.14 317 TrÇn ThÞ Ngäc Mai 6.47 6.59 0.02 318 NguyÔn Thµnh Nh 7.53 6.59 0.88 319 NguyÔn ThÞ Th¶o 6.67 6.59 0.01 320 Lª Minh Thanh 5.6 6.59 0.99 321 NguyÔn Thu Trang 7.27 6.59 0.46 322 NguyÔn ViÕt TuÊn 8.53 6.59 3.75 323 Vò ViÕt TuÊn 4.2 6.59 5.73 324 NguyÔn Anh V¨n 8.87 6.59 5.18 325 NguyÔn TrÇn ViÖt 7.73 6.59 1.29 326 Ng« ThÞ V©n Anh 7.6 6.59 1.01 327 Phïng Xu©n Chinh 6.33 6.59 0.07 328 NguyÔn ThÕ Dòng 6.6 6.59 0 329 NguyÔn TuÊn Dòng 3.13 6.59 12 330 Nh©m ThÞ Giang 8.27 6.59 2.81 331 TrÇn V¨n H¶i 7.13 6.59 0.29 332 NguyÔn ThÞ Hiªn 7 6.59 0.17 333 Ng« Duy Huynh 7 6.59 0.17 334 NguyÔn Thanh HuyÒn 8.07 6.59 2.18 335 NguyÔn Thanh HuyÒn 6.8 6.59 0.04 336 NguyÔn Ngäc Linh 7.6 6.59 1.01 337 Bïi Ngäc Long 5.47 6.59 1.26 338 Cao ThÞ Nhi 8 6.59 1.98 339 T« Minh Pha 9.13 6.59 6.43 340 NguyÔn §øc Phong 6.67 6.59 0.01 341 TrÇn v¨n Sang 5.2 6.59 1.94 APPENDIX 4 ITEM ANALYSIS OF THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1- 150 ITEMS, 349 EXAMINEES ITEM ANALYSIS FOR THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1- 150 ITEMS, 349 EXAMINEES Item No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Item correct 121 234 236 207 226 215 221 224 220 214 206 217 214 217 218 212 225 210 213 205 Item incorrect 228 115 113 142 123 134 128 125 129 135 143 132 135 132 131 137 124 139 136 144 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.35 0.67 0.68 0.59 0.65 0.62 0.63 0.64 0.63 0.61 0.59 0.62 0.61 0.62 0.62 0.61 0.64 0.6 0.61 0.59 Proportion incorrect (q) 0.65 0.33 0.32 0.41 0.35 0.38 0.37 0.36 0.37 0.39 0.41 0.38 0.39 0.38 0.38 0.39 0.36 0.4 0.39 0.41 Item variance 0.23 0.22 0.22 0.24 0.23 0.24 0.23 0.23 0.23 0.24 0.24 0.24 0.24 0.24 0.23 0.24 0.23 0.24 0.24 0.24 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 Item No 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Item correct 220 225 212 206 210 215 206 215 212 200 223 216 227 217 217 215 221 205 202 213 Item incorrect 129 124 137 143 139 134 143 134 137 149 126 133 122 132 132 134 128 144 147 136 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.63 0.64 0.61 0.59 0.6 0.62 0.59 0.62 0.61 0.57 0.64 0.62 0.65 0.62 0.62 0.62 0.63 0.59 0.58 0.61 Proportion incorrect (q) 0.37 0.36 0.39 0.41 0.4 0.38 0.41 0.38 0.39 0.43 0.36 0.38 0.35 0.38 0.38 0.38 0.37 0.41 0.42 0.39 Item variance 0.23 0.23 0.24 0.24 0.24 0.24 0.24 0.24 0.24 0.24 0.23 0.24 0.23 0.24 0.24 0.24 0.23 0.24 0.24 0.24 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 ITEM ANALYSIS FOR THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1- 150 ITEMS, 349 EXAMINEES Item No 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Item correct 202 215 200 221 211 208 225 214 222 221 259 244 244 238 234 239 236 231 223 225 Item incorrect 147 134 149 128 138 141 124 135 127 128 90 105 105 111 115 110 113 118 126 124 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.58 0.62 0.57 0.63 0.6 0.6 0.64 0.61 0.64 0.63 0.74 0.7 0.7 0.68 0.67 0.68 0.68 0.66 0.64 0.64 Proportion incorrect (q) 0.42 0.38 0.43 0.37 0.4 0.4 0.36 0.39 0.36 0.37 0.26 0.3 0.3 0.32 0.33 0.32 0.32 0.34 0.36 0.36 Item variance 0.24 0.24 0.24 0.23 0.24 0.24 0.23 0.24 0.23 0.23 0.19 0.21 0.21 0.22 0.22 0.22 0.22 0.22 0.23 0.23 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 Item No 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 Item correct 222 218 224 215 197 212 222 212 219 227 240 215 228 230 226 240 217 227 239 232 Item incorrect 127 131 125 134 152 137 127 137 130 122 109 134 121 119 123 109 132 122 110 117 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.64 0.62 0.64 0.62 0.56 0.61 0.64 0.61 0.63 0.65 0.69 0.62 0.65 0.66 0.65 0.69 0.62 0.65 0.68 0.66 Proportion incorrect (q) 0.36 0.38 0.36 0.38 0.44 0.39 0.36 0.39 0.37 0.35 0.31 0.38 0.35 0.34 0.35 0.31 0.38 0.35 0.32 0.34 Item variance 0.23 0.23 0.23 0.24 0.25 0.24 0.23 0.24 0.23 0.23 0.21 0.24 0.23 0.22 0.23 0.21 0.24 0.23 0.22 0.22 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 ITEM ANALYSIS FOR THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1- 150 ITEMS, 349 EXAMINEES Item No 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 Item correct 232 225 238 219 229 240 229 237 247 246 241 243 248 247 262 249 271 260 270 265 Item incorrect 117 124 111 130 120 109 120 112 102 103 108 106 101 102 87 100 78 89 79 84 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.66 0.64 0.68 0.63 0.66 0.69 0.66 0.68 0.71 0.7 0.69 0.7 0.71 0.71 0.75 0.71 0.78 0.74 0.77 0.76 Proportion incorrect (q) 0.34 0.36 0.32 0.37 0.34 0.31 0.34 0.32 0.29 0.3 0.31 0.3 0.29 0.29 0.25 0.29 0.22 0.26 0.23 0.24 Item variance 0.22 0.23 0.22 0.23 0.23 0.21 0.23 0.22 0.21 0.21 0.21 0.21 0.21 0.21 0.19 0.2 0.17 0.19 0.18 0.18 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 Item No 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 Item correct 260 263 260 267 260 258 246 241 246 215 219 221 223 201 198 224 200 214 212 229 Item incorrect 89 86 89 82 89 91 103 108 103 134 130 128 126 148 151 125 149 135 137 120 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.74 0.75 0.74 0.77 0.74 0.74 0.7 0.69 0.7 0.62 0.63 0.63 0.64 0.58 0.57 0.64 0.57 0.61 0.61 0.66 Proportion incorrect (q) 0.26 0.25 0.26 0.23 0.26 0.26 0.3 0.31 0.3 0.38 0.37 0.37 0.36 0.42 0.43 0.36 0.43 0.39 0.39 0.34 Item variance 0.19 0.19 0.19 0.18 0.19 0.19 0.21 0.21 0.21 0.24 0.23 0.23 0.23 0.24 0.25 0.23 0.24 0.24 0.24 0.23 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 ITEM ANALYSIS FOR THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1- 150 ITEMS, 349 EXAMINEES Item No 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 Item correct 207 212 222 224 221 219 229 239 228 224 222 241 242 240 240 242 258 255 267 260 Item incorrect 142 137 127 125 128 130 120 110 121 125 127 108 107 109 109 107 91 94 82 89 Total response 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.59 0.61 0.64 0.64 0.63 0.63 0.66 0.68 0.65 0.64 0.64 0.69 0.69 0.69 0.69 0.69 0.74 0.73 0.77 0.74 Proportion incorrect (q) 0.41 0.39 0.36 0.36 0.37 0.37 0.34 0.32 0.35 0.36 0.36 0.31 0.31 0.31 0.31 0.31 0.26 0.27 0.23 0.26 Item variance 0.24 0.24 0.23 0.23 0.23 0.23 0.23 0.22 0.23 0.23 0.23 0.21 0.21 0.21 0.21 0.21 0.19 0.2 0.18 0.19 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 Item No 141 142 143 144 145 146 147 148 149 150 Item correct 263 264 270 271 282 289 295 284 294 251 Item incorrect 86 85 79 78 67 60 54 65 55 98 Total response 349 349 349 349 349 349 349 349 349 349 Proportion correct (p) 0.75 0.76 0.77 0.78 0.81 0.83 0.85 0.81 0.84 0.72 Proportion incorrect (q) 0.25 0.24 0.23 0.22 0.19 0.17 0.15 0.19 0.16 0.28 Item variance 0.19 0.18 0.18 0.17 0.16 0.14 0.13 0.15 0.13 0.2 ∑pq 33 S.d of total scores 1.46 Variance of total scores 2.12 N (items) 150 N -1 149 KD-20 -14.6 APPENDIX 5 ITEM INDICES OF THE FINAL ACHIEVEMENT COMPUTER-BASED MCQS TEST 1 ITEM INDICES OF THE FINAL ACHIEVEMEMT COMPUTER-BASED MCQS TEST 1 Section Reading Item No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 p-value 0.35 0.67 0.68 0.59 0.65 0.62 0.63 0.64 0.63 0.61 0.59 0.62 0.61 0.62 0.62 0.61 0.64 0.6 0.61 0.59 item discrimination value 0.55 0.67 0.63 0.73 0.68 0.72 0.69 0.78 0.69 0.72 0.7 0.67 0.72 0.66 0.71 0.7 0.66 0.74 0.72 0.73 Section Reading Vocabulary Item No 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 p-value 0.63 0.64 0.61 0.59 0.6 0.62 0.59 0.62 0.61 0.57 0.64 0.62 0.65 0.62 0.62 0.62 0.63 0.59 0.58 0.61 item discrimination value 0.69 0.69 0.69 0.74 0.73 0.76 0.78 0.7 0.65 0.72 0.61 0.65 0.67 0.7 0.73 0.7 0.73 0.66 0.71 0.71 Section Vocabulary Item No 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 p-value 0.58 0.62 0.57 0.63 0.6 0.6 0.64 0.61 0.64 0.63 0.74 0.7 0.7 0.68 0.67 0.68 0.68 0.66 0.64 0.64 item discrimination value 0.73 0.7 0.68 0.69 0.69 0.73 0.68 0.74 0.7 0.73 0.63 0.67 0.64 0.63 0.62 0.7 0.64 0.63 0.66 0.68 Section Vocabulary Item No 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 p-value 0.64 0.62 0.64 0.62 0.56 0.61 0.64 0.61 0.63 0.65 0.69 0.62 0.65 0.66 0.65 0.69 0.62 0.65 0.68 0.66 item discrimination value 0.66 0.71 0.69 0.7 0.71 0.63 0.67 0.65 0.62 0.64 0.64 0.68 0.66 0.67 0.68 0.65 0.67 0.65 0.64 0.64 Section Grammar Item No 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 p-value 0.66 0.64 0.68 0.63 0.66 0.69 0.66 0.68 0.71 0.7 0.69 0.7 0.71 0.71 0.75 0.71 0.78 0.74 0.77 0.76 item discrimination value 0.69 0.7 0.61 0.69 0.66 0.69 0.72 0.76 0.67 0.67 0.67 0.69 0.63 0.65 0.63 0.66 0.65 0.65 0.64 0.63 Section Grammar Item No 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 p-value 0.74 0.75 0.74 0.77 0.74 0.74 0.7 0.69 0.7 0.62 0.63 0.63 0.64 0.58 0.57 0.64 0.57 0.61 0.61 0.66 item discrimination value 0.55 0.54 0.52 0.55 0.52 0.59 0.57 0.62 0.61 0.6 0.7 0.62 0.61 0.58 0.64 0.6 0.68 0.68 0.65 0.67 Section Grammar Functional Language Item No 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 p-value 0.59 0.61 0.64 0.64 0.63 0.63 0.66 0.68 0.65 0.64 0.64 0.69 0.69 0.69 0.69 0.69 0.74 0.73 0.77 0.74 item discrimination value 0.71 0.69 0.67 0.63 0.64 0.67 0.7 0.65 0.68 0.71 0.67 0.63 0.6 0.64 0.64 0.64 0.59 0.62 0.57 0.6 Section Functional Language Item No 141 142 143 144 145 146 147 148 149 150 p-value 0.75 0.76 0.77 0.78 0.81 0.83 0.85 0.81 0.84 0.72 item discrimination value 0.6 0.59 0.59 0.63 0.63 0.59 0.6 0.59 0.59 0.63