LOT Winter School 2019

RM1 - Reproducibility in bilingualism and SLA

Sible Andringa


Title of the course: Reproducibility in bilingualism and SLA

Teacher: Sible Andringa

Address: Spuistraat 134, 1012 VB, Amsterdam

Email address: s.j.andringa@uva.nl

website teacher: http://www.uva.nl/profiel/a/n/s.j.andringa/s.j.and...

Course info
Level: RM1 (First year Research Master Linguistics)

Course description:
The field of Cognitive Psychology is currently in a reproducibility crisis. In a recent, large replication effort, it was shown that the results of no more than 39 out of 100 studies were replicated (Open Science Collaboration, 2015). This raises questions about the value of the knowledge accumulated in this field. This begs the question: how reproducible are findings in bilingualism and second language acquisition? In this course, we will explore the major methodological challenges of our field. The articles prescribed were chosen to exemplify a particular challenge that is prevalent in hour field. In each meeting, we’ll discuss the article in relation to the methodological challenge at hand, and the extent to which the field may suffer from this problem and what that implies for what we know. The goal of this course is to foster a better understanding of how we can produce research on bilingualism and SLA that is more sustainable.

Day-to-day program

  • Monday: Theory-driven vs. data-driven research. The field of bilingualism and SLA tends to be rich in data. It’s easy to run a number of tests and look at how they are related. But can you interpret such data if you don’t have a theory that properly motivates your selection of tests?
  • Tuesday: Comparing groups. Monolinguals and bilinguals; natives and nonnatives, early learners and late learners… they are often compared, but comparing groups is not without problems. How valid are such comparisons?
  • Wednesday: Measurement and debriefing. When it comes to reproducibility, many people are concerned about data dredging and p-hacking, but the true problem may be more fundamental: what about the validity of the measures we choose? Do the measures we choose support the conclusions we draw?
  • Thursday: WEIRD science and reproducibility. As in many other behavioral fields, conclusions in the field of bilingualism and SLA are often based on WEIRD, non-representative samples. Do our conclusions hold beyond such samples?
  • Friday: Science for society. Impact and societal relevance is becoming increasingly important. Should research be driven by such concerns? And are researchers sufficiently equipped to address society’s concerns?

Reading list

Background and preparatory readings:

Marsden et al. (2018). Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field. Language Learning 68, 2, 321-390. https://doi.org/10.1111/lang.12286

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Course readings (obligatory):
(Subject to change)

Lecture 1:

Curcic, M., Andringa, S. J., & Kuiken, F. (2019). The Role of Awareness and Cognitive Aptitudes in L2 Predictive Language Processing. Language Learning, 69. https://doi.org/10.1111/lang.12321

Lecture 2:

Andringa, S. J. (2014). The use of native speaker norms in critical period hypothesis research. Studies in Second Language Acquisition, 36, 565-596. https://doi.org/10.1017/S0272263113000600

Lecture 3:

Andringa, S. J., & Curcic, M. (2015). How explicit knowledge affects online L2 processing: Evidence from differential object marking acquisition. Studies in Second Language Acquisition, 37, 237-268. https://doi.org/10.1017/S0272263115000017

Lecture 4:

Morgan-Short et al. (2018). Multisite Replication in Second Language Acquisition Research: Attention to Form During Listening and Reading Comprehension. Language Learning 68, 2, pp 392-437. https://doi.org/10.1111/lang.12292

Lecture 5:

Renner, F., Kersbergen, I., Field, M., & Werthmann, J. (2018). Dutch courage? Effects of acute alcohol consumption on self-ratings and observer ratings of foreign language skills. J Psychopharmacol, 32(1), 116-122. https://doi.or/10.1177/0269881117735687