Linguistics Colloquium Speaker: Marten van Schijndel

The Department of Linguistics proudly presents Dr. Marten van Schijndel, Professor at Cornell University. Dr. van Schijndel will speak on "Largely Language Modeling". Language modeling is an old task which has gained popularity as the backbone of the current AI boom. This is, in part, because the task is simple to implement at scale, and in part because the method has achieved good success at a number of language tasks. However, the learned representations of neural language models are opaque, making it difficult to assess what they encode. Many current discussions about language models center around their emergent properties; their apparent ability to transcend the language modeling task and instead achieve human-like language understanding. While language models work well for natural language processing tasks, I argue that language model representations differ in key ways from human linguistic representations. I discuss multiple studies from my group that identify weaknesses in these models and in the ways they are used which produce mismatches with human linguistic response patterns. While I provide methods for mitigating some of these weaknesses, caution should be employed when using these models since they are largely language models.