NAACL 2019 | Google BERT Wins Best Long Paper

The annual conference of the North American Chapter of the Association for Computational Linguistics (NAACL) is a grand event in the field of natural language processing. NAACL 2019 received 1198 long paper and 757 short paper submissions for a total of 1955 papers. The conference accepted 424 papers for a 22.6 percent acceptance rate.

This morning the NAACL 2019 conference committee announced its best paper awards. Synced has prepared a summary of the winning papers:

BEST LONG PAPER

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Authors: Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova (Google AI)
• Paper link: https://arxiv.org/pdf/1810.04805.pdf
• Abstract: We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models (Peters et al., 2018; Radford et al., 2018), BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT representations can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE benchmark to 80.4% (7.6% absolute improvement), MultiNLI accuracy to 86.7% (5.6% absolute improvement) and the SQuAD v1.1 question answering Test F1 to 93.2 (1.5 absolute improvement), outperforming human performance by 2.0.

BEST THEMATIC PAPER

• What’s in a Name? Reducing Bias in Bios Without Access to Protected Attributes
• Authors: Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky and Adam Kalai (CMU, Microsoft Research, LinkedIn)
• Paper link: https://128.84.21.199/abs/1904.05233
• Abstract: In the context of mitigating bias in occupation classification, we propose a method for discouraging correlation between the predicted probability of an individual’s true occupation and a word embedding of their name. This method leverages the societal biases that are encoded in word embeddings, eliminating the need for access to protected attributes. Crucially, it only requires access to individuals’ names at training time and not at deployment time. We evaluate two variations of our proposed method using a large-scale dataset of online biographies. We find that both variations simultaneously reduce race and gender biases, with almost no reduction in the classifier’s overall true positive rate.

BEST SHORT PAPER

• Probing the Need for Visual Context in Multimodal Machine Translation
• Authors: Ozan Caglayan, Pranava Madhyastha, Lucia Specia and Loïc Barrault (Le Mans University, Imperial College London)
• Paper link: https://arxiv.org/abs/1903.08678
• Abstract: In this paper we probe the contribution of the visual modality to state-of-the-art MMT models by conducting a systematic analysis where we partially deprive the models from source-side textual context. Our results show that under limited textual context, models are capable of leveraging the visual input to generate better translations. This contradicts the current belief that MMT models disregard the visual modality because of either the quality of the image features or the way they are integrated into the model.

BEST RESOURCE PAPER

• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• Authors: Alon Talmor, Jonathan Herzig, Nicholas Lourie and Jonathan Berant (Tel Aviv University, Alan Artificial Intelligence Research Institute)
• Paper link: https://arxiv.org/abs/1811.00937
• Abstract: To investigate question answering with prior knowledge, we present CommonsenseQA: a challenging new dataset for commonsense question answering. To capture common sense beyond associations, we extract from ConceptNet (Speer et al., 2017) multiple target concepts that have the same semantic relation to a single source concept. Crowd-workers are asked to author multiple-choice questions that mention the source concept and discriminate in turn between each of the target concepts. This encourages workers to create questions with complex semantics that often require prior knowledge. We create 12,247 questions through this procedure and demonstrate the difficulty of our task with a large number of strong baselines. Our best baseline is based on BERT-large (Devlin et al., 2018) and obtains 56% accuracy, well below human performance, which is 89%.

BEST EXPLAINABLE NLP PAPER

• CNM: An Interpretable Complex-valued Network for Matching
• Authors: Qiuchi Li, Benyou Wang and Massimo Melucci
• (Paper has yet to be published)

The NAACL 2019 will be held in Minneapolis, USA, from June 2 to June 7.

---

Author: Reina Qi Wan | Editor: Michael Sarazen