cross-lingual - ruder.io

ruder.io

Sign in Subscribe

cross-lingual

Posts about cross-lingual learning, mainly using cross-lingual word embeddings.

The State of Multilingual AI

The State of Multilingual AI

This post takes a closer look at the state of multilingual AI. How multilingual are current models in NLP, computer vision, and speech? What are the main recent contributions in this area? What challenges remain and how we can we address them?

Why You Should Do NLP Beyond English

Why You Should Do NLP Beyond English

7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.

10 ML & NLP Research Highlights of 2019

natural language processing

10 ML & NLP Research Highlights of 2019

This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019.

Unsupervised Cross-lingual Representation Learning

Unsupervised Cross-lingual Representation Learning

This post expands on the ACL 2019 tutorial on Unsupervised Cross-lingual Representation Learning. It highlights key insights and takeaways and provides updates based on recent work, particularly unsupervised deep multilingual models.

NAACL 2019 Highlights

NAACL 2019 Highlights

This post discusses highlights of NAACL 2019. It covers transfer learning, common sense reasoning, natural language generation, bias, non-English languages, and diversity and inclusion.

Neural Transfer Learning for Natural Language Processing (PhD thesis)

transfer learning

Neural Transfer Learning for Natural Language Processing (PhD thesis)

This post discusses my PhD thesis Neural Transfer Learning for Natural Language Processing and some new material presented in it.

The 4 Biggest Open Problems in NLP

natural language processing

The 4 Biggest Open Problems in NLP

This is the second post based on the Frontiers of NLP session at the Deep Learning Indaba 2018. It discusses 4 major open problems in NLP.

10 Exciting Ideas of 2018 in NLP

transfer learning

10 Exciting Ideas of 2018 in NLP

This post gathers 10 ideas that I found exciting and impactful this year—and that we'll likely see more of in the future. For each idea, it highlights 1-2 papers that execute them well.

EMNLP 2018 Highlights: Inductive bias, cross-lingual learning, and more

EMNLP 2018 Highlights: Inductive bias, cross-lingual learning, and more

This post discusses highlights of EMNLP 2018. It focuses on talks and papers dealing with inductive bias, cross-lingual learning, word embeddings, latent variable models, language models, and datasets.

Requests for Research

transfer learning

Requests for Research

It can be hard to find compelling topics to work on and know what questions to ask when you are just starting as a researcher. This post aims to provide inspiration and ideas for research directions to junior researchers and those trying to get into research.

Word embeddings in 2017: Trends and future directions

word embeddings

Word embeddings in 2017: Trends and future directions

Word embeddings are an integral part of current NLP models, but approaches that supersede the original word2vec have not been proposed. This post focuses on the deficiencies of word embeddings and how recent approaches have tried to resolve them.

Highlights of EMNLP 2017: Exciting datasets, return of the clusters, and more

natural language processing

Highlights of EMNLP 2017: Exciting datasets, return of the clusters, and more

This post discusses highlights of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017). These include exciting datasets, new cluster-based methods, distant supervision, data selection, character-level models, and many more.

A survey of cross-lingual word embedding models

A survey of cross-lingual word embedding models

Monolingual word embeddings are pervasive in NLP. To represent meaning and transfer knowledge across different languages, cross-lingual word embeddings can be used. Such methods learn representations of words in a joint embedding space.