AI Technology & Industry Review
A research team from Microsoft and NVIDIA leverages the NVIDIA Megatron-LM and Microsoft’s DeepSpeed to create an efficient and scalable 3D parallel system that combines data, pipeline, and tensor-slicing based parallelism, achieving superior zero-, one-, and few-shot learning accuracies and new state-of-the-art results on NLP benchmarks.
Microsoft releases DeBERTaV3, improving the original DeBERTa model using ELECTRA-style pretraining with gradient-disentangled embedding sharing to achieve better pretraining efficiency and a significant performance jump.
In the paper Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers, a research team from New York University and the University of North Carolina at Chapel Hill uses centered kernel alignment (CKA) to measure the similarity of representations across layers and explore how fine-tuning changes transformers’ learned representations.
A research team from Baidu proposes ERNIE 3.0, a unified framework for pretraining large-scale, knowledge-enhanced models that can easily be tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning, and achieves state-of-the-art results on NLP tasks.
A research team from Google Brain and New York University says the Natural Language Understanding (NLU) evaluation system is “broken” and proposes four criteria for improving NLU benchmarks.
A new model surpassed human baseline performance on the challenging natural language understanding benchmark.
In EMNLP 2020, a Montreal-based research team introduced a large medical text dataset designed to boost abbreviation disambiguation in the medical domain.
Google has now released a major V2 ALBERT update and open-sourced Chinese ALBERT models.
Chinese AI company iFLYTEK has bested the SQuAD2.0 challenge once again. The model “BERT + DAE + AoA” submitted by the joint iFLYTEK Research and HIT (Harbin Institute of Technology) laboratory HFL outperformed humans on both EM (exact match) and F1-score (fuzzy match) indexes to top the SQuAD2.0 leaderboard.
Earlier this week the Association for Computational Linguistics (ACL) 2018 announced its Best Two Short Papers, neither of which had yet been published. Today the AI community got its first look at one of the winners when Know What You Don’t Know: Unanswerable Questions for SQuAD was released on arXiv.
People care most about having good content and being recognized by virtual assistants, the AI-powered human-machine conversational interfaces.
Google and Amazon unveiled mini-sized smart speakers Google Home Mini Chalk and Amazon Echo Dot 2, both priced at less than US$50. Virtual assistants can plug into your environment with a natural human-machine voice interface that used to exist only science fiction.