Liulishuo’s AI App Is Teaching English to 70 Million People

“Liulishuo is the AI English teacher on your phone. You don’t need to know how it works, yet it helps you learn English more efficiently than a human teacher,” says Yi Wang, Founder and CEO of Liulishuo — a Beijing-based “AI + language” company whose name translates as “speak fluently.” Liulishuo hosts the world’s largest speech bank of Chinese speakers speaking English.

After obtaining a PhD in computer science from Princeton, Wang worked as a product manager at Google in California. He returned to China in the early 2010s and found many of his friends asking similar questions: “How do I learn English?”; “Why is it that I pay so much money for lessons and fail to keep up?”; “Should I watch more American TV shows to learn to speak naturally?”

Wang wondered whether people might be able to learn English by speaking with their phones. At the time, AI and online education were much less developed. In 2012 Wang launched Liulishuo with Hui Lin, a Google coworker specializing in voice recognition and machine learning.

Launched six months ago, the company’s flagship app is now teaching English with personalized and adaptive methods based on deep learning to some 70 million registered users in 175 countries. It was selected as an Apple Store “Best App of the Year,” and Apple VP Philip Schiller and his team visited the company’s offices. Liulishuo is the only Chinese Education company to make the CB Insights AI 100 list.

Synced recently spoke with Liulishuo Yi Wang about his company and AI language learning.

How does Liulishuo apply AI to its teaching products?

Liulishuo has the world’s largest database of Chinese people speaking English. Based on the database we created a Chinese-speaking-English recognition engine with the highest accuracy and an evaluation engine which provides users with ratings and feedback.

Initially, we used speech recognition to evaluate the user’s verbal skills. Now our engine can perform a full range of verbal language assessments, and we have a separate engine that can correct writing.

We have a special function tackling the IELTS exams, which tests users on their pronunciation, grammar, vocabulary, and fluency. The scoring algorithms have passed the Turing Test. The variation in score between our AI and a human examiner is lower than between two human examiners. We offer users individualized suggestions to improve their English using their scores in the four areas.

We launched the world’s first AI English teacher platform in July 2016. Ten million users have already completed our ranking exam, with completion time ranging from 5 minutes to 20 minutes depending on proficiency.

After completing the exam, the system selects a starting level for each user. Users improve their English through the immersive learning of scenes from images and animations, without any subtitles or translations. They must attempt to understand the scenario, which is followed by suggested practices, which is then followed by more progressive scenarios. This back and forth is highly effective in improving the user’s English language skills.

We have the data to prove our effectiveness. ETS TOEFL testing has proven our AI teacher can improve learning efficiency by three times. For example, if it used to require 100 hours of learning to reach a certain level in the CEFR standard, we would only need about 36. Regardless of product and service, we’re the only English learning organization that publishes efficiencies of this caliber, and that really excites me.

Where did you get your first set of voice recognition data?

Before launching the product, we asked some American English speakers to record audio for us to cold start the engine. We also collected limited data from Chinese speakers audio recordings via crowdsourcing.

But since the launch of our product, our users have provided us with massive amounts of incoming data in different skill levels and accents. This data was recorded by users reading what’s displayed on their screen, and so it is also labeled. We effectively received all this data for free from users practicing their English.

Where does the evaluation engine get its standard voice data?

In order to train our model, we invited experts to label our data for us, for example for IELTS we asked IELTS examiners to label our data.

Where did Liulishuo get content like animations, short videos, and scenarios?

We have two types of content. The free content is English conversations written by professional writers, on top of User-Generated Content (UGC) and Professionally Generated Content (PGC). We also have the most active English learning community in China, and many of our short videos are contributed by these learners.

The paid content is created by our own team. We hired Philip Lance Knowles, who has previously proposed Recursive Hierarchical Recognition Theory and other breakthroughs in language learning theory based on cognitive science, as our expert consultant and created our content based on Lance’s theory. Of course, our customized learning material is different from writing a textbook, where all students learn in the same sequence.

Did you ever think of using AI to create content?

This is the general direction, and we are doing some early stage testing.

Many companies are working on translation headphones, such as those presented at the Google I/O event. As AI translation advances, will we still need to learn English in the future?

These headphones can benefit let’s say seniors traveling in another country. But I think they won’t be able to replace the language learning market. Firstly, the Ministry of Education will not remove English from the curriculum just because we have translation machines. Secondly, learning a new language isn’t just about translation, translation is just application.

Learning a new language is about learning to communicate with others, and there are cultural contexts one must also learn to understand in order to learn real communication. In the process of learning a new language, there’s a sense of accomplishment, in which the user builds confidence and challenges themselves. Thus we see the social function of language learning, as many people learn to make new friends. There are people who make friends and even find their partner on our platform. In this sense, you can’t equal language learning with translation.

We want the learning process to be customized and highly effective. To achieve these two goals, we believe that data-driven AI is the key. We’ve only taken the first step in exploring AI teachers. They aren’t intelligent enough just yet, and the learning experience has many areas that can still be optimized. We are working hard at solving these problems.

People have limited understanding of how our conscience and brain actually work. We are working with many experts in neuroscience and education, such as the Dean of Education Faculty at Stanford University and a Professor of Neurology at Yale University, in hopes of bringing in new research results. Our platform is also useful for their research because we have large amounts of user data that helps them create new learning models. We have set up an education and AI lab in the Bay Area, hoping to attract top experts in AI, education, and cognitive science, in order to help us create the most intelligent, most efficient AI English teacher in the world.

What were some turning points for Liulishuo?

The explosive growth of the mobile internet since 2012 has turned mobile device usage into the new way of life. I saw this as an immense opportunity, but I realized that if I only developed small apps focusing on weather, calendar, camera etc, it would be a challenge to make them profitable. Therefore we thought about combining the mobile internet with traditional industries. The markets must be large, with good user paying habits, and room for improving efficiency. We researched applications in finance, health, and education, finally deciding to go with education.

We set forth to create an easy-to-use product. The first week Liulishuo was available, it was recommended by the Apple Store in mainland China, Hong Kong, Taiwan and Japan, and quickly became one of their “Best Apps of the Year.” Apple’s Senior VP of Worldwide Marketing soon toured our company. This shows that our product-centered strategy is being rewarded.

The second turning point was the transformation of Liulishuo from a tools App into a community App. Building a community increased user stickiness and activity and created a broad learning environment.

The third point was in 2014, when we made a strategic decision to create an educational research team, to involve education professionals from a content perspective. Before this, we were purely an internet company. We decided to put a heavy focus on the essence of education and personalization of content. If we had not made that shift, we would just be a marginalized tech company.

Were there any detours?

In 2014 we worked with a foreign company and tested two learning techniques which used word games to practice speaking. But they weren’t very successful. From a gameplay perspective, they weren’t as fun as normal games; while from a learning perspective, they weren’t very effective.

During the second half of 2014 we wanted to create a textbook product. Our first instinct was to license the best textbooks from top publishers such as Pearson, Cambridge, or Oxford. After we got to know them a bit better, we realized that these textbooks were written and designed for traditional, offline classroom scenarios and were not effective for users lounging on their couch learning with a smartphone. Therefore, we started to work on our own educational research team and spent two years developing the AI teacher. This was a strategic change, and looking back it was the right thing to do.

What trends do you see in English language learning?

We are seeing three historical opportunities. Firstly, learning is now digitized. In the past we did our homework on paper, whereas today 100% of user learning, actions, and interactions are digitized. This is a huge leap forward and the only way to make it possible to use AI. If you’re not digitized and have no structured data, it will be impossible to talk of AI, right?

Secondly, I think we’re experiencing a historic leap from teacher-centric to student-centric learning. There were many more students than teachers in the industrial era, but many students are now practicing one-on-one with a language tutor. However, this is a transition phase because the so-called “one-on-one” is still not necessarily centered around the student, as teachers may not understand the needs of the student and create suitable teaching plans.

Liulishuo’s AI teacher is not human, it is a system that relies on user interactions to make decisions. Through deep learning and other AI technologies, it selects only relevant content from a huge library and recommends it based on the student’s level. I think the pace of review and practice frequency based on an individual student’s needs, strengths and weaknesses is the ultimate student-centric learning experience.

Third, from a business perspective, we believe that we must develop a results-oriented business model to replace a process-oriented one. In a language training institution for example, if you learn for 100 hours and yet still see no improvement, the institution won’t be responsible as it has delivered its service by selling you the teacher’s time. Hence, these traditional training institutions are just wholesalers of teachers’ time. We think this situation will eventually change, and educators will get paid according to the results achieved by each student.

Our paid product works exactly this way, it does not charge based on instructional hours, but instead, provides users with a buffet. For just CN¥99 a month, users can spend as much time as they like there. A diligent student can learn at a much faster pace, absorbing all that they can. Our paid users on average spend five hours or more learning on our App each week. Who spends five hours a week learning a new skill anymore as an adult? This shows our product is really effective.

What is Liulishuo’s short-term and long-term plan?

I hope that in the next two to three years Liulishuo can assemble a leading team of researchers and product designers with full-stack development capabilities, dedicated to applying AI to education. As for long-term plans, I hope that in the next decade, we can become a global leader in education.