Elite Chinese Music School Now Offers PhDs in Music + AI

On March 21 Google Magenta and PAIR celebrated Johann Sebastian Bach’s 334th birthday with an interactive AI-powered Doodle that transforms user-input notes into harmonized melodies in the German composer’s style.

AI has been exploring its musical side over the last year: Facebook AI Research introduced its universal music translation network, which leverages a multi-domain WaveNet autoencoder to generate impressive music translations from a human-whistled input melody; Google Magenta proposed the intelligent 8-button controller interface “Piano Genie” which maps user improvisations onto a full 88-keyboard piano performance; and Google researcher Jesse Engel and his team introduced efficient, high-quality audio synthesis with their generative model GANSynth.

Now, China’s elite Central Conservatory of Music (CCOM) has announced it is recruiting PhDs for a new Music AI and Information Technology program. CCOM says prospective students should have a background in Computer Science, AI, or Information Technology; along with musical abilities (instrument playing or singing). As course preparation, the school recommends four books on AI: Data Structure and Algorithms, Signals and Systems, Artificial Intelligence: A Modern Approach, and Neural Networks and Learning Machines; and a single music-related book, Fundamentals of Music Theory.

CCOM Dean Feng Yu will serve as a music training advisor; while two top Chinese AI scholars will act as mentors: Vice Dean of Tsinghua University Institute for AI Professor Maosong Sun; and Vice Dean of the School of Electronics Engineering and Computer Science at Peking University Professor Xihong Wu.

This nexus of Music + AI continues to capture the imagination of computer scientists. Bach was also celebrated in the seminal CS book Gödel, Escher, Bach: An Eternal Golden Braid, published in 1979. Forty years later the question remains: can a computer create beauty?

Stanford University offers both PhD and Master’s programs in computer-based music theory and acoustics through the Center for Computer Research in Music and Acoustics (CCRMA), and a Master’s program in Music, Science, and Technology (MST). Carnegie Mellon University offers both Bachelor’s and Master’s of Science in Music and Technology and has a Computer Music Project focusing on music theory, Cognitive Science, AI and Machine Learning. Simon Fraser University in Canada has its Metacreation Lab, where researchers explore AI in creative tasks such as music composition, sound design and audio effect generation. The Kyoto University Graduate School of Informatics Department of Intelligence Science and Technology has a cutting-edge Speech and Audio Processing Laboratory. The Vienna University of Technology’s Institute of Software Technology and Interactive Systems and its Information & Software Engineering Group conduct research on music information retrieval with Machine Learning.

CCOM showcased Music + AI at its November 2018 event An Evening of Classical Music and AI. Professional musicians performed solos on flute, violin, cello, bassoon, trumpet, clarinet, viola, saxophone, oboe, horn, trombone and erhu — with live “orchestral” accompaniment provided by AI-powered software application Cadenza Live.

Based on a new Informatics Philharmonic System, Cadenza Live is the result of a collaboration between CCOM and the Indiana University School of Informatics, Computing and Engineering. The two institutions jointly founded the Informatics Philharmonic Orchestra Laboratory in May 2018.

Indiana University Professor Christopher Raphael, who developed the Informatics Philharmonic System, explains that the system first listens to and interprets audio input from a live soloist using a Hidden Markov Model. It then uses a Kalman Filter-like model to predict and schedule its accompaniment according to the tempo and style of the player, and finally generates the accompaniment audio by phase-vocoding prerecordings of accompaniment-only tracks using its prediction model. The system continuously modifies model parameters to improve performance.

Last December Beijing’s Central University for Nationalities (CUN) and AI company Ping An Technology founded a Joint Laboratory of AI Music that aims to leverage AI in musical applications and enhance CUN’s scientific research capabilities in Music + AI.

The Conference on Sound and Music Technology (CSMT) is an annual gathering first hosted by Tsinghua University and Fudan University in 2013 to showcase interdisciplinary music studies. Topics at the 6th CSMT last November included Virtual Human and AI in Speech and Music Processing and AI algorithms for Automatic Composition of Music. There was also a special session discussing the fusion of traditional Chinese music with Computer Science and frontier technologies.

Localization: Tingting Cao | Editor: Michael Sarazen