Last week, Tencent Graph Computing (TGraph) officially announced the release of Plato, an open source high-performance graph computing framework that meets the ultra-large-scale graph computing requirement of billion-level nodes. Plato can shorten algorithm computing time from days to just minutes and reduce the number of servers required to complete a task from hundreds to only about ten — feats unattainable by any other mainstream distributed graph computing framework.
Graph computing combines data from different sources and of different kinds into the same graph to find correlations and connections which are difficult to discern through distinct data analysis approaches. Graph computing is increasingly used as a data analysis and mining tool across social networks and recommendation systems, as well as in the cyber security, text retrieval and biomedical fields.
The main contributions of the Plato framework are:
- On ultra-large-scale social network graph data, its performance has reached top levels in academia and industry, exceeding Apache’s popular graph and graph-parallel computation tool Spark GraphX by at least 10 times on calculation.
- Plato consumes significantly less memory compared to Spark GraphX, ranging from 16 times smaller to 116 times smaller on different benchmarks, which means a middle-to-small-scale cluster with as little as ten servers can do a calculation, greatly reducing compute costs.
- As a part of TGraph, Plato originates from ultra-large-scale social network graph data, but its adaptive graph computing engine can also perfectly accommodate other graph data types.
Plato can provide efficient offline graph computing and graph representation learning for social network data on the massive scale produced by Tencent. It runs on general X86 clusters, such as Kubernetes and Yarn clusters, and supports multiple interfaces for mainstream file systems such as HDFS and Ceph.
At the Tech Echo Developer Conference earlier this month Tencent also officially announced the open-sourcing of four other key projects: TubeMQ, Tencent Kona JDK, TBase, and TKEStack. So far Tencent has open sourced 86 projects on GitHub, ranking among the leading software-sharing platform’s top 10 global corporate contributors.
The Plato Graph Computing Framework, installation instructions and further information is available on GitHub.
Author: Reina Qi Wan | Editor: Michael Sarazen