CMU Multilingual NLP - Dependency Parsing

Explore dependency parsing, its applications, and cross-lingual methods in this lecture from CMU's Multilingual Natural Language Processing course. Delve into linguistic structure types, Universal Dependencies Treebank, and techniques for adding inductive bias to neural models. Examine practical applications like extracting morphological agreement rules and analyzing linguistic phenomena in film scripts. Learn about arc standard shift-reduce parsing, graph-based dependency parsing, and the Chu-Liu-Edmonds algorithm. Discover sequence model feature extractors and biaffine classifiers for parsing tasks. Address challenges in multilingual dependency parsing through order-insensitive encoders, generative model fine-tuning, and linguistically informed constraints. Gain insights into improving cross-lingual transfer and handling structural differences between languages.

Two Types of Linguistic Structure
Why Dependencies?
Universal Dependencies Treebank Standard format for parse trees in many languages
Adding Inductive Bias to Neural Models • Bias self attention to follow syntax
Understanding Language Structure Example of extracting morphological agreement rules using dependency relations
Searching over Parsed Corpora Search using 'syntactic regex'
Analysis of Other Linguistic Phenomena • Examining power and agency in film scripts
Arc Standard Shift-Reduce Parsing (Yamada & Matsumoto 2003, Nivre 2003)
Shift Reduce Example
Classification for Shift-reduce
Encoding Stack Configurations w/ RNNS
Transition-based parsing State embeddings
(First Order) Graph-based Dependency Parsing
Graph-based vs. Transition Based
Chu-Liu-Edmonds (Chu and Liu 1965, Edmonds 1967)
Find the Best Incoming
Subtract the Max for Each
Recursively Call Algorithm
Expand Nodes and Delete Edge Deleted from cycle
Sequence Model Feature Extractors (Kipperwasser and Goldberg 2016)
BiAffine Classifier (Dozat and Manning 2017)
Difficulty In Multilingual Dependency Parsing
Example Improvement 1: Order-insensitive Encoders . Standard cross-lingual transfer can fail with large word order differences between source and target Change model structure to be order-insensitive to avoid over-titting to source
Generative Model Fine-tuning • Use generative model that can be trained unsupervised, and fine-tune on the target language
Example Improvement 3: Linguistically Informed Constraints • Add constraints based on a priori knowledge of the language structure
Discussion Question