AI Machine Learning & Data Science Research

IBM Leverages Reinforcement Learning to Achieve SOTA Performance on Text and Knowledge Base Generation

In the paper ReGen: Reinforcement Learning for Text and Knowledge Base Generation Using Pretrained Language Models, IBM researchers present ReGen, a bidirectional generation of text and graph that leverages reinforcement learning to push the performance of text-to-graph and graph-to-text generation tasks to a higher level.

Knowledge bases (KBs) can be used to store complex structured and unstructured information, and are a powerful tool for capturing real-world information with complex relationships. Automatic KB generation from free-form text and the generation of semantically meaningful text from KBs are crucial and challenging research areas in machine learning.

In the new paper ReGen: Reinforcement Learning for Text and Knowledge Base Generation Using Pretrained Language Models, an IBM research team proposes ReGen, a method for bidirectional generation of text and graph that leverages reinforcement learning (RL) to push the performance of text-to-graph (T2G) and graph-to-text (G2T) generation tasks to a higher level.

The team summarizes their main contributions as:

  1. Propose using RL-based sequence training — specifically SCST — for both G2T and T2G tasks. This is the first time that RL-based training has been proposed for the bi-directional generation of text and graphs.
  2. Demonstrate that our approach provides better performance than the best systems reported for the WebNLG 2020+ Challenge.
  3. Provide a thorough investigation of SCST-based training for both T2G and G2T tasks, including best rewards combination.
  4. Construct subject and relation-object boundaries from TEKGEN sentence-triples pairs and show the performance of our approach for both T2G and G2T tasks.
  5. Adapt the large-scale TEKGEN corpus (Agarwal et al., 2021) for T2G and G2T tasks and confirm the benefit of the SCST-based fine-tuning approach over CE-trained baselines.

Inspired by the idea that sequence generation tasks can be re-framed such that a model chooses the best word within a given vocabulary, the team reformulates Seq2Seq generation into the RL framework. Simply put, an agent will define a policy that selects each word during the generation procedure. In this way, the model is employed as its own critic, enabling self-critical sequence training (SCST).

The team used T5 PLMs – t5-large (770M parameters) and t5-base (220M parameters) for their evaluation experiments. The models were fine-tuned to be either specialized on T2G (MT) or G2T (MG) tasks, or to accommodate both generation directions (MT+G).

The team reported results on the WebNLG+ 2020 (v3.0) datasets in the WebNLG 2020 Challenge, which comprises two tasks: RDF-to-text generation (G2T) and text-to-RDF semantic parsing (T2G). They also used the TEKGEN dataset to test the robustness of their proposed system, using BLEU, BLEU_NLTK, METEOR, and chrF++ as evaluation metrics.

The results demonstrate that the use of RL via SCST benefits graph and text generation on WebNLG+ 2020 and TEKGEN datasets, significantly improving on previously published results to achieve state-of-the-art performance on WebNLG+ 2020 for both text-to-graph and graph-to-text generation tasks.

In their future work, the researchers plan to develop an SCST variant that better leverages the unique structure of graphs — either by performing more sensible graph-dependent sampling or by investigating alternative reward schemes more attuned to the integration of the content and structure of graphs.

The paper ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Models is on arXiv.


Author: Hecate He | Editor: Michael Sarazen, Chain Zhang


We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.

3 comments on “IBM Leverages Reinforcement Learning to Achieve SOTA Performance on Text and Knowledge Base Generation

  1. Pingback: r/artificial - [R] IBM Leverages Reinforcement Learning to Achieve SOTA Performance on Text and Knowledge Base Generation - Cyber Bharat

  2. Thanks for sharing. It will definitely be interesting to many. And it’s really inspiring. Our training system is outdated for a long time, I want all this to be introduced into training now. I believe that services such as https://writehistoryessay.com/pay-someone-to-write-history-essay-proficiently/, which provide quality historical essay writing services, are also important especially for successful learning.

  3. While the most important thing is to find a good and reliable place to help with writing competent essays and homework. We have picky teachers, so I’m glad I was able to find someone to turn to and make our work top notch. I found Essaygeeks.co.uk here and so far they haven’t let me down once. That’s why I highly recommend you to check them out!

Leave a Reply

Your email address will not be published.

%d bloggers like this: