Fork me on GitHub

Character-based recurrent neural networks for morphological relational reasoning

Character-based recurrent neural networks for morphological relational reasoning. The <em>FC relation</em> layer is connected to an auxilliary output layer, trained to predict a label for the current type of relation. The final output is generated by the <em>Decoder RNN</em>.

We present a model for predicting inflected word forms based on morphological analogies. Previous work includes rule-based algorithms that determine and copy affixes from one word to another, with limited support for varying inflectional patterns. In related tasks such as morphological reinflection, the algorithm is provided with an explicit enumeration of morphological features which may not be available in all cases. In contrast, our model is feature-free: instead of explicitly representing morphological features, the model is given a demo pair that implicitly specifies a morphological relation (such as write:writes specifying infinitive:present). Given this demo relation and a query word (e.g. watch), the model predicts the target word (e.g. watches). To address this task, we devise a character-based recurrent neural network architecture using three separate encoders and one decoder. Our experimental evaluation on five different languages shows that the exact form can be predicted with high accuracy, consistently beating the baseline methods. Particularly, for English the prediction accuracy is 95.60%. The solution is not limited to copying affixes from the demo relation, but generalizes to words with varying inflectional patterns, and can abstract away from the orthographic level to the level of morphological forms.

Preliminary version appeared in Subword & Character Level Models in NLP (SCLeM) workshop at EMNLP 2017 in Copenhagen, Denmark, September 7.

Source code

The source code used for the experiments can be downloaded from https://github.com/olofmogren/char-rnn-wordrelations.

Olof Mogren, Richard Johansson

Journal of language modelling.
bibtex.

Olof Mogren, PhD, RISE Research institutes of Sweden