LOT Winter School 2018

RM1 Robot writers: Computational models of text generation

Emiel Krahmer & Albert Gatt


Tilburg University & University of Malta


Course info

Level: RM1

Course description:

Robot writers are computer programs that can automatically produce coherent natural language text based on input data. They have their roots in the field of Natural Language Generation (NLG), which is a subfield of artificial intelligence and computational linguistics. The goal of this course is to introduce robot writing for students and researchers interested in linguistics, both as an interesting application domain for theories of language production and as a research tool for studying human language production.

While the basic technology required for building robot writers has been around for some time now, the quickly growing volumes of available data and the increase of digital publishing in recent years has paved the way for this new application, which allows for fast, large scale, personalised, and on-demand information presentation. In this course, we provide a brief, non-technical introduction to robot writers, discussing their history, development and evaluation.

Special attention will be devoted to the use of robot writers as a model for human language production, comparing the design of a typical system with the proposed blueprint for the speaker. We also highlight recent developments related to image-to-text description (i.e., robot writers which generate description of pictures) and to creative language (i.e., robot writers which generate new metaphors or poetic language).

After following this course, students (1) know what the problems and prospects of robot writers are, (2) understand the benefits of these systems for a better understanding of human language production, and (3) have hands-on experience with evaluating the performance of state-of-the-art text generation applications.

Day-to-day program

Monday: Introduction to robot writers / Comparison with human language production / Theory and applications [EK]

Tuesday: Production of creative language / Theories of creativity and their applications to text generation / metaphor, narrative and poetry [AG]

Wednesday: Computational psycholinguistics of text generation / Models of human reference production [EK]

Thursday: Vision-to-language / state of the art neural models[AG]

Friday: Evaluation of robot writers [AG/EK]

Reading list

Background and preparatory readings:

Gatt, A., & Krahmer, E. (2017). Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation. arXiv:1703.09902 [cs.CL]

This survey covers much of what will be discussed during this course. It is a draft and all comments/suggestions are welcome and appreciated.

Course readings:

For every lecture, two texts have been selected, the first providing a general overview and the second presenting a somewhat more detailed case study. Students are asked to select one from each pair, depending on their interests. All texts can be found on Google scholar.

Lecture 1:

  • Reiter, E., & Dale, R. (1997). Building applied natural language generation systems. Natural Language Engineering, 3(1), 57-87.

  • Levelt, W. J. (1999). Producing spoken language: A blueprint of the speaker. In The neurocognition of language (pp. 83-122). Oxford University Press.

Lecture 2:

  • Gervas, P. (2009). Computational approaches to storytelling and creativity. AI Magazine, 49–62.
  • Colton, S., Goodwin, J., & Veale, T. (2012). Full-FACE Poetry Generation. In Proceedings of the International Conference on Computational Creativity (ICCC) (pp. 95-102)

Lecture 3:

  • Van Deemter, K., Gatt, A., Van Gompel, R. P., & Krahmer, E. (2012). Toward a computational psycholinguistics of reference production. Topics in cognitive science, 4(2), 166-183.
  • Frank, M. C., & Goodman, N. D. (2012). Predicting pragmatic reasoning in language games. Science, 336(6084), 998-998.

Lecture 4:

  • Bernardi, R., Cakici, R., Elliott, D., Erdem, A., Erdem, E., Ikizler-Cinbis, N., Keller, F., Muscat, A., & Plank, B. (2016). Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures. Journal of Artificial Intelligence Research (JAIR), 55, 409-442.
  • Mitchell, M., Han, X., Dodge, J., Mensch, A., Goyal, A., Berg, A., Yamaguchi, K., Berg, T., Stratos, K., & Daumé III, H. (2012). Midge: Generating image descriptions from computer vision detections. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 747-756).

Lecture 5:

  • Mellish, C., & Dale, R. (1998). Evaluation in the context of natural language generation. Computer Speech & Language, 12(4), 349-373.
  • Reiter, E., & Belz, A. (2009). An investigation into the validity of some metrics for automatically evaluating natural language generation systems. Computational Linguistics, 35(4), 529-558.

Further readings (optional):