A research team from Microsoft and NVIDIA leverages the NVIDIA Megatron-LM and Microsoft’s DeepSpeed to create an efficient and scalable 3D parallel system that combines data, pipeline, and tensor-slicing based parallelism, achieving superior zero-, one-, and few-shot learning accuracies and new state-of-the-art results on NLP benchmarks.
In the paper Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers, a research team from New York University and the University of North Carolina at Chapel Hill uses centered kernel alignment (CKA) to measure the similarity of representations across layers and explore how fine-tuning changes transformers’ learned representations.
A research team from Baidu proposes ERNIE 3.0, a unified framework for pretraining large-scale, knowledge-enhanced models that can easily be tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning, and achieves state-of-the-art results on NLP tasks.
Chinese AI company iFLYTEK has bested the SQuAD2.0 challenge once again. The model “BERT + DAE + AoA” submitted by the joint iFLYTEK Research and HIT (Harbin Institute of Technology) laboratory HFL outperformed humans on both EM (exact match) and F1-score (fuzzy match) indexes to top the SQuAD2.0 leaderboard.
Earlier this week the Association for Computational Linguistics (ACL) 2018 announced its Best Two Short Papers, neither of which had yet been published. Today the AI community got its first look at one of the winners when Know What You Don’t Know: Unanswerable Questions for SQuAD was released on arXiv.
Google and Amazon unveiled mini-sized smart speakers Google Home Mini Chalk and Amazon Echo Dot 2, both priced at less than US$50. Virtual assistants can plug into your environment with a natural human-machine voice interface that used to exist only science fiction.