California Startup Builds Worlds Largest AI Chip; Open AI Releases 774M Parameter Language Model; Waymo Shares Its Open Dataset

Subscribe to Synced Global AI Weekly

Hot Chips | A California Startup Has Built an AI Chip as Big as a Notebook. Why?
California AI startup Cerebras Systems has introduced its Cerebras Wafer Scale Engine (WSE), the world’s largest-ever chip built for neural network processing. The 16nm WSE is a 46,225 mm2 silicon chip featuring 1.2 trillion transistors, 400,000 AI optimized cores, 18 Gigabytes of on-chip memory, 9 petabyte/s memory bandwidth, and 100 petabyte/s fabric bandwidth.
(Synced)

Open AI Releases 774 Million Parameter GPT-2 Language Model
Open AI releases the 774 million parameter GPT-2 language model after the release of their small 124M model in February, the staged release of a medium 355M model in May, and subsequent research with partners and the AI community into the model’s potential for misuse and societal benefit.
(Open AI)

Waymo Open Dataset: Sharing Self-Driving Data for Research
Waymo is sharing the Waymo Open Dataset, a high-quality multimodal sensor dataset for autonomous driving. Waymo researchers believe it is one of the largest, richest, and most diverse self-driving datasets ever released.
(Waymo)

Technology

Turbo, An Improved Rainbow Colourmap for Visualization
A research team from Google AI has introduced Turbo, a new colourmap that has the desirable properties of Jet while also addressing some of its shortcomings, such as false detail, banding and colour blindness ambiguity. Turbo was hand-crafted and fine-tuned to be effective for a variety of visualization tasks.
(Google AI )

On The Variance of The Adaptive Learning Rate and Beyond
Researchers propose RAdam, a new variant of Adam, by introducing a term to rectify the variance of the adaptive learning rate. Extensive experiment results on image classification, language modeling, and neural machine translation demonstrate the effectiveness and robustness of the proposed method.
(UIUC & Georgia Tech & Microsoft)

A Critique of Pure Learning and What Artificial Neural Networks Can Learn from Animal Brains
Researchers argue that most animal behavior is not the result of clever learning algorithms — supervised or unsupervised — but rather is encoded in the genome. Specifically, animals are born with highly structured brain connectivity, which enables them to learn very rapidly.
(N ature)