In a blog post today OpenAI today announced the final staged release of its 1.5 billion parameter language model GPT-2, along with all associated code and model weights.
GPT-2 is a large language model that can generate realistic paragraphs of text. Since its debut this February, GPT-2 has attracted both praise for its performance and concerns regarding its potential misuse for example in generating fake news stories. The latter fears initially prompted Open AI to take the unusual step of withholding the language model’s code and training dataset. Over the past nine months, OpenAI researchers have made the following observations regarding GPT-2:
1. Humans find GPT-2 outputs convincing.
2. GPT-2 can be fine-tuned for misuse.
3. Detection is challenging.
4. We’ve seen no strong evidence of misuse so far.
5. We need standards for studying bias.
Even with no task-specific training data, GPT-2 still reaches state-of-the-art performance on machine translation, question answering, reading comprehension, and summarization.
When the GPT-2 model debuted in February, OpenAI told Synced it was withholding the model’s code and training dataset: “it’s clear that the ability to generate synthetic text that is conditioned on specific subjects has the potential for significant abuse.” OpenAI’s decision came under fire in the AI community, as did the company’s decision the following month to launch a new, for-profit division, OpenAI LP. The San Francisco-based research organization started in 2015 as a non-profit company with a goal of achieving a safe and socially beneficial artificial general intelligence. The company said its for-profit division was a necessary move to balance huge compute and AI talent costs, but some, like software engineer Brian Baliat, suspected an ulterior motif: “I now see why you didn’t release the fully trained model of #gpt2.”
OpenAI says it is now releasing the GPT-2 model to “aid the study of research into the detection of synthetic text, although this does let adversaries with access better evade detection.”
The AI community reacted quickly to today’s PT-2 release. Within an hour, machine learning engineer Adam King had updated his GPT-2 powered interactive text generating website: “The ‘too dangerous to release’ GPT-2 text generator is finally fully released! I’ve put it up on http://TalkToTransformer.com for those who like to live dangerously.”
The GPT-2 output detection model is available on GitHub, and OpenAI’s report on potential malicious use can be downloaded in pdf.
Journalist: Fangyu Cai | Editor: Michael Sarazen
Really an interesting blog I have gone through. There are excellent details you posted here. Sometime it is not so easy to design and develop a AI and Machine Learning project without custom knowledge; here you need proper development skill and experience. However the details you mention here would be very much helpful for the beginner.
Know more about Machine Learning and AI: Artificial Intelligence