AI is booming. So is the compute used to train AI models. San Francisco-based non-profit AI research institute OpenAI yesterday released an analysis on the largest AI models since 2012. Researchers discovered that the amount of compute used for training doubles every 3.5 months. By comparison, the number of transistors per square inch on integrated circuits only doubles every 18 months, a.k.a Moore’s Law.
The analysis also showed that DeepMind’s 2017 AlphaZero, which mastered the game of Go after mere hours of training with no human game records to reference, ran over 300,000 times more compute than AlexNet, the breakthrough GPU-powered convolutional neural network (CNN) that won the 2012 ILSVRC (ImageNet Large-Scale Visual Recognition Challenge).
The results suggest that more compute can achieve better performance in AI models, and is often complementary to algorithmic advances.
The analysis also suggests that the exponential increase of compute for AI training can be largely attributed to hardware development. Before 2012, AI researchers had merely co-opted GPUs for machine learning, but after 2016 specialized processors such as TPUs and more advanced interconnect techniques that enable large-scale parallel computing allowed researchers to train AI models with better compute capability.
OpenAI researchers adopted two methods to measure the compute directly counting operations used in a forward pass, or calculating operations based on numbers of GPUs deployed for training. For example, an AlexNet paper revealed that the network takes “between five and six days to train on two GTX 580 3GB GPUs”, so researchers could estimate an approximate amount of compute of 0.0058 petaflops per day.
Synced collected comments from AI scholars and journalists on OpenAI’s new analysis:
I’m not sure you can conclude that AI capabilities will always advance in step with compute power. That assumes all you need is more data and GPU horsepower.
— Will Knight, MIT Technology Review’s Senior Editor for Artificial Intelligence.
I think it’s totally backwards. Techies love playing with bigger toys, so they’ll try to use whatever they can get their hands on. The chart is showing the increasing availability of resources to DL researchers — that’s all.
— Jeremy Howard, Previously Kaggle President and Chief Scientist, and now Fast.ai Founding Researcher
If there is an underlying technique (architecture search, self-play, …) which can utilize vast compute effectively for a given real task, the amount of compute available for this purpose is astronomical.
— Stephen Merity, Previously Salesforce Senior Research Scientist
This chart via @OpenAI is amazing on its own, but the most interesting part imo is how much of an outlier DQN remains.
— Dave Gershgorn, Quartz Reporter for Artificial Intelligence
Journalist: Tony Peng| Editor: Michael Sarazen
Your article is such an informative article. It is glad to read such those articles thanks for sharing. Akira Hoodie
So AI master games now. What happens when they master humans.
thanks for this article. comic con costumes for sale
Thanks for this Article its very useful for me.
Athena Karkanis Manifest S03 Black Jacket
Such a wonderful Article its very good keep it up.
Manifest Josh Dallas Shirt Style Leather Jacket
thanks for this artical.
Melissa Roxburgh Manifest Brown Wool Coat
I must say this post is really great and informative too, keep sharing more I really like the content. While exploring things if there is any technical issue.
Thank you for posting such a great article! I found your website perfect for my needs.
Are you looking forward to getting your hands on something unique? Visit our website to see our wide range of stylish and fashionable jackets, including Gaming jackets, Biker jackets, Leather jackets, and more. Simply click the link to make a purchase.
Astonishing! What a by and large differentiation to be consolidated here and to be significant for this thoroughly shocking neighborhood people. In case someone let me in on a half year earlier I would be in the position I am today by essentially taking a course called Connect With Anyone, I would never have confided in it
AI very hard site. This AI information benefits people. This is a very nice post.
I must mention that this post is both interesting and educational. please continue to share as I enjoy the information. Are you excited to get your hands on something special? Then Elvis Austin Butler Leather Jacket is for you. The best leather apparel for your winter collection.
This article is outstanding in every way. Thank you for providing this useful information.
biker jacket
Yes article about Artificial intelligence, or AI, involves using computers and algorithms to mimic human intelligence, revolutionizing the way we work and live. I work in
best tow truck companies in queens county i saw your website and there are many of biker jackets i like to purchase because i love to wear jacket when i am driving on the highways.
This article yellowstone jackets is very amazing.Thank for sharing this post.
google great one keep it up
I’m checking out your site for the first time, and I must say that it’s great and very beneficial for kids. You’re doing an outstanding job, and I’ll be returning soon. famous ladies jackets Good luck in the future.
Is it accurate to say that you are searching for something special In the event that truly, at that point head to gatekeepers to get the most moving outfits at truly reasonable rates? The nature of the items is very good quality which will show up more appealing at any event and function’ Thanksgiving suit for men
I can’t stop playing Flappy Bird. It makes me happy. Thanks for this great game.
I’m visiting your website for the first time, and I must say that it’s fantastic and very useful for children. You’re doing an excellent job, and I’ll be back shortly. Pelle Pelle Jacket
It’s enjoyable to read such lovely and instructive material. I enjoyed reading the useful content on your website. Please keep sharing with us.
Jack Harlow Varsity Jacket
Great Blog. The Best Wi-Fi SMS app is TextNow, which allows users to send unlimited text messages over Wi-Fi for free. It also includes features like group messaging, voice messaging, and multimedia sharing.
I enjoy your blog. Great blog, keep sharing this type of blog again I visit this type of blog. I share this Poker Face Natasha Lyonne Brown Leather Jacket is constructed of real leather, and the viscose lining inside provides warmth and comfort.
I’m visiting this website for the first time I’m very happy to say that your blog is wonderful it’s a lot of time I spend on this website I will come again to this website.
This is really a great website, i suggest you to post articles to attract visitors attention. Your website is really a great source of information.Snoop Dogg Super Bowl Halftime Tracksuit
I appreciate you sharing this fantastic article. I adore your article. This website is very user friendly and well done. Thank you for contributing your ideas. Jensen Ackles Jacket Please feel free to browse my web.
Wonderful article. Fascinating to read. I love to read such an excellent article. Thanks! It has made my task more and extra easy. Keep rocking.
Yellowstone Merch
I appreciate the research and effort you put into creating this content.
Walter White Jacket
Your ideas and info is really unique and you clear my all doubts about this topic. I also want to share some thing unique try this Attack On Titan Eren Jeager Cosplay Black Hooded Coat
Those who grew up with their favourite media franchises will love the touch of celebrity attire, such as this
Half Shark Camo BAPE Hoodie worn by TV actress, singer. Jacket of a famous person
AI has come a long way in the era that is going on at the moment which is also helping a lot of people. I also have to write content about a Star Trek Picard Season 3 Deanna Troi Leather Jacket for my siteso I take the help of ai. And I enjoy reading your ai related content.
I searched a lot about AI on the net, but I did not find any content like yours. AI is very useful, so everyone searches about AI on the net. Now I will give you some fashion information regarding winter. In this winter season, I will advise you to use this And Just Like That S02 Carrie Bradshaw Plaid Coat
It is the best time to upgrade your look and feel free to get this amazing Johnny Lawrence Red Cobra Kai Jacket. the leather-fabricated outerwear is just best to make you stand out in any event.
Smoke wholesale Texas refers to the thriving business of supplying various smoking products and accessories in the state of Texas. Texas, known for its rich barbecue culture and appreciation for smoking meats, has a robust market for wholesale smoking products. Wholesalers in Texas offer a wide range of items, including premium cigars, tobacco, rolling papers, hookahs, and vaping supplies.
Wow its a very good post.
NewYork Jacket
Very nice Article Keep it up Squid Game Season 2 Tracksuit
Keep sharing good information
Superheroes Costumes
You are absolutely right that AI is on the rise. And if AI is used for a specific purpose, that’s something we can use. Like the Karlie Kloss Red Bomber Jacket has a special purpose that makes you look beautiful and fashionable.
AI is the biggest and most advanced technology of this era. And indeed AI is on the rise. And if AI is used in a positive way, it can be very good. Now I will suggest you to use this A Royal Christmas Crush 2023 Stephen Huszar Puffer Coat in this cold season.
the example you have provided really help illustrate your points effectively. I appreciate the effort you have put into researching and gathering relevant information as well vape distributor texas
The best faux leather jackets are the ones that make you happy. And our effort is to provide you with leather faux shearling jacket. So visit our site now.
Great post! Easy to understand and super informative. for more you can also go for the best breast nipple shield
thank you
Yo, let’s talk about VictoriyaClub—this site’s like the VIP section of safe online dating services! They’ve got your back with top-notch security and legit profiles, so you can swipe and chat without any sketchy stuff. Have you had any sketchy experiences with other dating sites? How do you stay safe while online dating? Let’s share some tips and tricks for navigating the online dating world safely!!
You have a good point here! I totally agree with what you have said!! Thanks for sharing your views.
You have a good point here! I totally agree with what you have said!! Thanks for sharing your views. Ghostbusters Frozen Empire Jacket
We are always with you and Waiting for your new interesting articles Ghostbusters Frozen Empire Jacket
Thanks for providing such blogs. Ghostbusters Frozen Empire Jacket
Extremely go through article. Chicago Jacket
DTF transfers, short for Direct-to-Film transfers, are a type of printing method used to transfer designs directly onto various surfaces, particularly fabrics. In dtf transfer sheets, the design is first printed onto a special film using a compatible printer and ink. Once the design is printed, it is transferred onto the desired substrate using a heat press or similar equipment.
Seamless integration with other productivity tools and platforms enables holistic data management and collaboration. Office 365 Backup
We follow the newest trends in fashion and are quite curious about clothing. The best website for leather clothing is Jacket Ware.