From May – July 2012, I faced one of the most important phases of my PhD project: conducting the experimental study in two ESL classrooms and collecting quantitative as well as qualitative data. In Applied Linguistics, quantitative data can be obtained in numerous ways; the most practical and widely used instrument is the language test. Before the experimental study, I thoroughly analysed different instruments and weighed the advantages and disadvantages of various test and task designs. Language testing and assessment are complex topics and the extensive literature with varying definitions and terms for the same phenomenon doesn’t make it easy to keep track of important information.
In my case, the instruments used to collect data included a designed language test comprising three different test tasks (a response task type, i.e. multiple choice, a limited production task, i.e. a gap-filling task, and an extended production task, i.e. a short essay), a designed questionnaire consisting of 28 questions (Likert-type scale, multiple answers), learners’ written assignments, observation notes (I attended the classrooms as a non-participant observer) and semi-structured interviews with the English teachers. The language test, which is the primary instrument, aimed at eliciting the learners’ performance concerning the English tense and aspect system, which means the learners’ grammatical ability in actual language use, was measured. Thus, the performance and yielded scores indicate how well learners know the required grammatical structure. The validity of the language test is ensured by two methods: content-related evidence (expert judgment) and construct-related evidence. Internal consistency is measured with the Kuder-Richardson approach.
Looking at the theory and knowledge acquired during the last year, I felt quite confident and well prepared to conduct the experiment. Well, up to the moment I entered the classrooms! Working with young learners, at the age of 12 to 15 years, challenged me as well as the experienced English teachers to a great extent. The majority of students were not too enthusiastic to participate in the experimental study, which was reflected in their insufficient motivation, disrespectful and impudent manners during the test/questionnaire administration and procedure, and their negative and repellent attitude towards school, classmates, teachers, and the subject English. I was pretty much shocked about the “reality” in schools, what being a teacher in secondary schools implicates (e.g. social work; to overcome the language barrier with foreign learners; etc.) and how marginal research can become in everyday school life. I was brave enough to stay the course and leave the fieldwork without any serious damage!
One month after leaving the fieldwork, I can say that conducting research with human beings, especially teenagers, does implicate events or situations you could not predict, calculate or research beforehand. It seems to me that applied linguists aren’t able to control all unrelated and extraneous variables all the time – I am aware of this now.
Cook, T., Campbell, D. (1979). Quasi-Experimentation. Design & Analysis Issues for Field Settings. Chicago: Rand Mc Nally College Publishing Company.
Creswell, J.W. (2009). Research Design. Qualitative, Quantitative, and Mixed Methods Approaches. 3rd Ed. Thousand Oaks: SAGE Publications.
Dörnyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University Press.
Fraenkel & Wallen (2000). How to Design & Evaluate Research in Education. Boston: McGrawHill.
Johnson, K. (2008). Quantitative Methods In Linguistics. Oxford: Blackwell Publishing.
Purpura, J. (2004). Assessing Grammar. Cambridge: Cambridge University Press.
Reif (2012). Making Progress Simpler? Applying Cognitive Grammar to Tense-Aspect Teaching. Frankfurt: Peter Lang.