Fancy a Friendly Chat? Stanford NLP’s Chirpy Cardinal Enables Open-Domain and Humanlike Conversations

The ongoing deployment of deep neural networks in the field of natural language processing (NLP) has enabled much more efficient AI agents for a variety of practical applications in customer service, etc. But sometimes people — for example, seniors or those experiencing social isolation — would just like to enjoy a friendly chat. However, building an open-domain social chatbot with a rich personality and an understanding of social dynamics that can carry on casual and engaging conversations with human beings remains challenging.

In the new paper Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent, a Stanford NLP research team presents Chirpy Cardinal, an open-domain conversational social chatbot with emotional and social intelligence that enables authentic interactions with real people. In last September’s Alexa Prize Socialbot Grand Challenge, Chirpy Cardinal placed second out of nine bots with an average user rating of 3.58/5.

The team summarizes their main contributions as follows:

1. Conversations with open-domain socialbots often lack a stable structure. To ameliorate this, we present an extensible design for open-domain dialogue which prioritizes conversational stability and flexibility through mixed initiative (Horvitz, 1999).
2. Although pretrained neural generators (Collins and Ghahramani, 2021) can be extremely fluent, real-life deployment can suffer from a lack of both controllability and consistency (Nie et al., 2021). Towards this end, we describe several approaches to integrate neural generation into a symbolic setup, achieving local fluency without sacrificing global coherence.
3. Towards the goal of a rewarding conversation, we suggest a set of approaches—ranging from small routines to complete submodules— which aim to make our socialbot a good conversational partner. We focus on being both flexible—handling a wide variety of topics in an interesting and informative way—and personable—empathizing with the other interlocutor even in difficult topics or situations.

The proposed Chirpy Cardinal is a novel conversational socialbot that combines traditional dialogue tree-based approaches with large pretrained neural dialogue agents. The team introduces an extensible design for open-domain dialogue that enables Chirpy Cardinal to fluently cover thousands of conversational topics. Every user dialogue is treated as a series of subconversations, and a response generator (RG) is employed to handle each of them. The system can seamlessly switch RGs in response to flighty prompts from the user and move to a new subconversation in a relatively seamless and humanlike manner.

The researchers also integrate neural generation in the context of hand-written scaffolding to better handle open-domain dialogues and maintain fluency and coherency as the conversation evolves over time, and distill a single model from BlenderBot-3B to achieve a significant latency reduction.

To enable emotional intelligence, the researchers introduce an Opinion RG which Chirpy Cardinal uses to solicit users’ views on topics and reciprocate with its “own” emotional tendencies. A Personal Issues RG is also incorporated to make the conversations more intimate and empathic.

The team evaluated Chirpy Cardinal on the Alexa Socialbot Grand Challenge 4, where it proved capable of dealing with thousands of conversations daily and earned second place with an average user rating of 3.58/5.

The team had open-sourced the Chirpy Cardinal system and provided a demo on the real-world deployment of conversational NLP systems. They hope their work can serve as a starting point and inform further socialbot development.

The code is available on the project’s GitHub. The paper Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent is on arXiv.

---

Author: Hecate He | Editor: Michael Sarazen

---

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.