Neural Transfer Learning for Natural Language Processing (PhD thesis)

I finally got around to submitting my thesis. The thesis touches on the four areas of transfer learning that are most prominent in current Natural Language Processing (NLP): domain adaptation, multi-task learning, cross-lingual learning, and sequential transfer learning.

Most of the work in the thesis has been previously presented (see Publications). Nevertheless, there are some new parts as well. The most notable are:

1. a background chapter (§2) that lays out key concepts in terms of probability and information theory, machine learning, neural networks, and NLP and connects these to their usage in subsequent chapters;
2. a taxonomy for transfer learning for NLP (§3.1.3, see below) that adapts the taxonomy of Pan and Yang (2010) to contemporary settings in NLP;
3. an updated review of multi-task learning (§3.2) that discusses more recent advances and choices in multi-task learning;
4. reviews of sequential transfer learning and domain adaptation (§3.3 and §3.4) that identify common themes in each of the research areas;
5. and future directions (§8.3) in each area that particularly excite me.

Whenever possible, I've tried to draw connections between methods used in different areas of transfer learning. It's a longer read but I hope it may still be helpful to some of you. You can download the complete thesis here.

If you found some material in the thesis helpful, I'd appreciate if you could cite it using the below BibTex:

@PhdThesis{Ruder2019Neural,
  title={Neural Transfer Learning for Natural Language Processing},
  author={Ruder, Sebastian},
  year={2019},
  school={National University of Ireland, Galway}
}