The ability of large language models to generate computer code from natural language (NL) prompts has revolutionized the programming domain. Most contemporary models however can only generate code for seen libraries and function calls, and struggle when they encounter any of the new libraries or functions that are constantly being introduced. A human programmer facing such a challenge would typically research and retrieve user manuals and other relevant documents to familiarize themselves with the new library/function — could LLMs be taught to do the same?
In the new paper DocPrompting: Generating Code by Retrieving the Docs, a research team from Carnegie Mellon University and Inspired Cognition presents DocPrompting, a novel NL-to-code generation approach. Tasked with generating code to unseen functions or libraries from an NL intent, DocPrompting retrieves corresponding code documentation to enable the model to learn to perform the task.
DocPrompting is inspired by programmers’ use of manuals and documentation when encountering unseen/unused functions or libraries. The approach first learns to retrieve relevant documents from an external documentation pool, then learns to generate code using prompts based on the information it gleaned from the documents.
The documentation pool can be regularly updated with new content to enable DocPrompting to generate unseen and unused functions and libraries without requiring any costly retraining of model components. DocPrompting is also a general method — it can be applied to any programming language and is not bounded to the underlying neural model, and can be instantiated with any base retriever and generator.
In their empirical study, the team evaluated DocPrompting on two NL-to-code tasks and benchmarks: shell scripting and Python programming. In the shell scripting task, DocPrompting consistently improved on the base model; while In Python programming, CodeT5+DocPrompting performed exceptionally well on unseen functions and achieved a 1.65 BLEU score improvement over the state-of-the-art result.
This work opens a promising new direction for the evolution of code generation. The team says that, to their best knowledge, DocPrompting is the first approach to explicitly and effectively leverage documentation for NL-to-code tasks.
The code is available on the project’s GitHub. The paper DocPrompting: Generating Code by Retrieving the Docs is on arXiv.
Author: Hecate He | Editor: Michael Sarazen
We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.
Your writing is perfect and complete.
From chill beats to high-energy jams, Sprunki Incredibox has it all! Each round is like creating a new song. Love it!
Game blends music and gameplay in a way that few games can—one mistake and it’s game over!
Hello! I love all the styles you share!
Wow, this is fascinating! The idea of LLMs learning to use documentation like human programmers is a real game-changer. It reminds me a little of playing Doodle Baseball – you don’t always nail the perfect hit right away, but you learn from each swing, adjusting your timing and strategy. Similarly, DocPrompting seems to allow the model to “swing” at new code challenges, learn from its “misses” by referencing the documentation, and ultimately improve its “hitting” average. The fact that it’s adaptable to different languages is super promising too. Thanks for sharing this!
papa’s games is a series of restaurant management simulation games where players take on the role of chefs and manage different food outlets.
Papa’s Games is a series of restaurant management simulation games where players take on the role of chefs and manage different food outlets.
Kontext Dev is your go-to AI image editor when you need precise local adjustments and consistent character details across edits. Start with a free trial and see how Kontext Dev combines multi-modal input and advanced control to help artists and developers achieve professional results with less effort.
HEIC to JPG – Fast & Easy HEIC Image Converter
this is so helpful for me . thanks!
I recently tried Hunty Zombie — it delivers spooky thrills, vivid visuals, and heart-racing scenes. The pacing is excellent, keeping tension high. The sound design adds an extra layer of immersion. Perfect for horror fans who enjoy being terrified and intrigued in equal measure.
Create original and royalty-free songs with DeepSong.ai now! – a free online AI-powered platform for fast, high-quality song & music generator.
Struggling with complex image edits? Kontext Dev offers free AI-powered local image editing with character consistency and multi-modal input — perfect for artists and developers.
Brat Generator makes it easy to design custom Charli XCX brat cover style images in seconds, free and online.
This is super interesting! LLMs struggling with new libraries is a real pain point, so improving code generation with documentation retrieval sounds like a game-changer. It’s cool how different types of data can solve such varied problems, from coding to even figuring out fandom stats in a fun game like ao3dle!
DocPrompting sounds like a huge win for LLMs and code generation, especially with new libraries! It’s cool how innovation makes everything better. That even applies to personal wellness. I’ve been looking into the best red light therapy devices for better sleep and recovery. So many advancements!
This is super interesting! Improving LLMs to handle new libraries is a game-changer for code generation, really making them more robust. It reminds me how important good documentation is. Sometimes, after intense coding, I need a different kind of ‘generation’ – like a fun break. Ever tried retro bowl 26 for a quick game? It’s a blast!
I recently explored Love Type — it’s a surprisingly soothing experience that mixes playful personality insights with a clean, elegant design. The questions feel intuitive, the results are fun to read, and the overall vibe is uplifting. Great for anyone who enjoys self-discovery with a light romantic twist.
Test your fandom knowledge with Rule34dle – the addictive daily game where you predict which characters have more Rule 34 content! Challenge yourself with 10 daily rounds or go infinite, compete with friends on global leaderboards, and discover fascinating trends in Rule 34 culture. Perfect for casual fans and dedicated enthusiasts alike. Play now and see how well you really know your favorite characters.
I recently checked out Hubble Birthday — it’s an engaging tool that pairs your birth date with a stunning Hubble photo, giving you a glimpse into the cosmos in a uniquely personal way. The design is clean, the results load quickly, and each image feels like a little snapshot of universal wonder. Perfect for space lovers or anyone curious about the universe’s timeline.
I recently explored The Forge Script — the site keeps things minimal and straightforward, making it easy to skim through the script features available for the game. The categories are clear, the functions are laid out simply, and the overall experience feels quick and practical.
z-img.art is an AI image generation and editing platform for creators, allowing them to quickly generate high-quality images simply by providing input prompts.
Such a useful explanation saved me a lot of time. Thanks!
nano banana pro is an AI image generation tool to create stunning images from prompts in seconds.