natural language processing

Posts about all areas of natural language processing.

cross-lingual

The State of Multilingual AI

This post takes a closer look at the state of multilingual AI. How multilingual are current models in NLP, computer vision, and speech? What are the main recent contributions in this area? What challenges remain and how we can we address them?

events

ACL 2022 Highlights

This post discusses my highlights of ACL 2022, including language diversity and multimodality, prompting, the next big ideas and keynotes, my favorite papers, and the hybrid conference experience.

natural language processing

Challenges and Opportunities in NLP Benchmarking

Over the last years, models in NLP have become much more powerful, driven by advances in transfer learning. A consequence of this drastic increase in performance is that existing benchmarks have been left behind. Recent models "have outpaced the benchmarks to test for them" (AI Index Report 2021)

language models

Recent Advances in Language Model Fine-tuning

This article provides an overview of recent methods to fine-tune large pre-trained language models.

transfer learning

ML and NLP Research Highlights of 2020

This post summarizes progress in 10 exciting and impactful directions in ML and NLP in 2020.

cross-lingual

Why You Should Do NLP Beyond English

7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.

natural language processing

10 ML & NLP Research Highlights of 2019

This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019.

cross-lingual

Unsupervised Cross-lingual Representation Learning

This post expands on the ACL 2019 tutorial on Unsupervised Cross-lingual Representation Learning. It highlights key insights and takeaways and provides updates based on recent work, particularly unsupervised deep multilingual models.

transfer learning

The State of Transfer Learning in NLP

This post expands on the NAACL 2019 tutorial on Transfer Learning in NLP. It highlights key insights and takeaways and provides updates based on recent work.

events

EurNLP

The first European NLP Summit (EurNLP) will take place in London on October 11, 2019. It is an opportunity to foster discussion and collaboration between researchers in and around Europe.

transfer learning

Neural Transfer Learning for Natural Language Processing (PhD thesis)

This post discusses my PhD thesis Neural Transfer Learning for Natural Language Processing and some new material presented in it.

events

AAAI 2019 Highlights: Dialogue, reproducibility, and more

This post discusses highlights of AAAI 2019. It covers dialogue, reproducibility, question answering, the Oxford style debate, invited talks, and a diverse set of research papers.

natural language processing

The 4 Biggest Open Problems in NLP

This is the second post based on the Frontiers of NLP session at the Deep Learning Indaba 2018. It discusses 4 major open problems in NLP.

transfer learning

10 Exciting Ideas of 2018 in NLP

This post gathers 10 ideas that I found exciting and impactful this year—and that we'll likely see more of in the future. For each idea, it highlights 1-2 papers that execute them well.

events

EMNLP 2018 Highlights: Inductive bias, cross-lingual learning, and more

This post discusses highlights of EMNLP 2018. It focuses on talks and papers dealing with inductive bias, cross-lingual learning, word embeddings, latent variable models, language models, and datasets.

natural language processing

HackerNoon Interview

This post is an interview by fast.ai fellow Sanyam Bhutani with me. It covers my background, advice on getting started with NLP, writing technical articles, and more.

language models

A Review of the Neural History of Natural Language Processing

This post expands on the Frontiers of Natural Language Processing session organized at the Deep Learning Indaba 2018. It discusses major recent advances in NLP focusing on neural network-based methods.

natural language processing

ACL 2018 Highlights: Understanding Representations and Evaluation in More Challenging Settings

This post discusses highlights of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018). It focuses on understanding representations and evaluating in more challenging scenarios.

natural language processing

NLP's ImageNet moment has arrived

Big changes are underway in the world of NLP. The long reign of word vectors as NLP's core representation technique has seen an exciting new line of challengers emerge. These approaches demonstrated that pretrained language models can achieve state-of-the-art results and herald a watershed moment.

natural language processing

Tracking the Progress in Natural Language Processing

Research in ML and NLP is moving at a tremendous pace, which is an obstacle for people wanting to enter the field. To make working with new tasks easier, this post introduces a resource that tracks the progress and state-of-the-art across many tasks in NLP.

natural language processing

Highlights of NAACL-HLT 2018: Generalization, Test-of-time, and Dialogue Systems

This post discusses highlights of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018). It focuses on Generalization, the Test-of-Time awards, and Dialogue Systems.

tensorflow

Text Classification with TensorFlow Estimators

This post is a tutorial that shows how to use Tensorflow Estimators for text classification. It covers loading data using Datasets, using pre-canned estimators as baselines, word embeddings, and building custom estimators, among others.

word embeddings

Word embeddings in 2017: Trends and future directions

Word embeddings are an integral part of current NLP models, but approaches that supersede the original word2vec have not been proposed. This post focuses on the deficiencies of word embeddings and how recent approaches have tried to resolve them.

multi-task learning

Multi-Task Learning Objectives for Natural Language Processing

Multi-task learning is becoming increasingly popular in NLP but it is still not understood very well which tasks are useful. As inspiration, this post gives an overview of the most common auxiliary tasks used for multi-task learning for NLP.

natural language processing

Highlights of EMNLP 2017: Exciting datasets, return of the clusters, and more

This post discusses highlights of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017). These include exciting datasets, new cluster-based methods, distant supervision, data selection, character-level models, and many more.