Learnability and language acquisition

Alexander Clark

Contact

Department of Computer Science,
Royal Holloway University of London,
Egham, TW20 0EX, United Kingdom

alexc@cs.rhul.ac.uk

http://www.cs.rhul.ac.uk/home/alexc

Course level

intermediate

Course description

As Chomsky observed, the fundamental problem of linguistics is to satisfy two conflicting goals: to find formalisms that are sufficiently expressive to represent the sorts of structures and dependencies that we observe in natural language, and secondly to account for their acquisition. This course analyses this problem using the tools of modern computational learning theory, surveying the classical literature on learnability, and how this motivated a rich notion of UG in the Principles and Parameters program, and moving on to the current movement in the Minimalist Program to reduce the work done by the language faculty, perhaps to a single primitive operation, MERGE. Along the way we will look at programs and algorithms for learning language, in both the theoretical sense, and practical implemented algorithms for doing unsupervised learning from real corpora, including some of child directed speech. (Much of this is joint work with Shalom Lappin).

Day-to-day program

Monday

The basic goals of linguistics: methodological issues. The tension between expressive power and learnability; Possible solutions. UG and Universal Grammar. The Minimalist Program and Biolinguistics. Approaching UG from above and from below.

Tuesday

Expressive power; arguments that language is not regular. Mild context sensitivity; Swiss German. Current problems: Yoruba, Old Georgian, Chinese number names. Bottom up versus top down derivations and the idea of MERGE.

Wednesday

Learnability. Gold's theorem and the Principles and Parameters program. Parameter setting algorithms. Problems with the P and P models.

Thursday

Distributional learning. American structuralism, and the Russian set-theoretical school. Angluin's learning of regular languages. Extending this result to context free and context sensitive languages.

Friday

Empirical work on unsupervised learning. Datasets and the choice of corpora; Intrinsic and extrinsic evaluations. Summary.

Reading materials

Background and preparatory readings

Alexander Clark and Shalom Lappin: “Linguistic Nativism and the Poverty of the Stimulus”, Wiley-Blackwell, (2011)

Course readings

Lecture 1: Chomsky (2005) Three factors in language design, Linguistic Inquiry. Chomsky, Noam (2007). “Approaching UG from below”. In: Sauerland, U.; Gaertner, H.-M. (eds.). Interfaces + recursion = language?

Lecture 2: Chomsky (1956) Three models for the description of language. IRE Transactions on Information Theory. Stuart Shieber (1985),Evidence against the context-freeness of natural language. Linguistics and Philosophy, 1985.

Lecture 3: Pinker, S. (1979). Formal Models of Language Learning. Cognition. 7, 217-283., Gold, E. M. (1967) Language identification in the limit. Information and Control, 10(5):447--474

Lecture 4:Dana Angluin. 1982. Inference of Reversible Languages. J. ACM 29, 3 (July 1982), 741-765. , Alexander Clark and Rémi Eyraud. 2007. Polynomial Identification in the Limit of Substitutable Context-free Languages. J. Mach. Learn. Res. 8 (December 2007), 1725-1745.

Lecture 5: John Goldsmith (2001) Unsupervised Learning of the Morphology of a Natural Language, Computational Linguistics, Alexander Clark (2003) Combining Distributional and Morphological Information for Part of Speech Induction, Proceedings of the EACL, Dan Klein and Chris Manning (2004) Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency, Proceedings of the ACL.