AI Technology & Industry Review
In a new paper Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2, a Google DeepMind research team introduces Gemma Scope, a comprehensive suite of JumpReLU SAEs.
In a new paper SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs, a research team from Google Research and Carnegie Mellon University introduces Semantic Pyramid AutoEncoder (SPACE), the first successful method for enabling frozen LLMs to solve cross-modal tasks.
The Swiss Federal Institute of Technology Lausanne (EPFL) presents Multi-modal Multi-task Masked Autoencoders (MultiMAE), a simple and effective pretraining strategy that enables masked autoencoding to include multiple modalities and tasks and is applicable to any RGB dataset.
The developers behind TabNine have introduced a new deep learning model, “Deep TabNine,” which significantly improves suggestion quality.
PixelGAN is an autoencoder for which the generative path is a convolutional autoregressive neural network on pixels, conditioned on a latent code, and the recognition path uses a generative adversarial network (GAN) to impose a prior distribution on the latent code.
New autoencoder-like generative network, called Adversarial Generator-Encoder Networks (AGE Network), does not have any discriminators, which makes the entire architecture much simpler than some recently-proposed GANs, but with nearly the same-level performance