When the steel mills shut down, America’s Rust Belt cities struggled to deal with stagnant economies, stale industries, and deteriorating infrastructures. Its strength in education and healthcare was all that saved Pittsburgh, Pennsylvania from bankruptcy.
Some 30 years later, buzzwords like “maker”, “tech”, and “innovation” scroll across screens at Pittsburgh International Airport, heralding the city’s rebirth as one of the most important AI hubs in America.
At the centre of Pittsburgh’s revival is Carnegie Mellon University, ranked #6 worldwide in computer science by Times Higher Education. The university recently announced a new artificial intelligence R&D initiative, “CMU AI,” which knits together more than 100 faculty members and 1,000 students, kickstarting a new chapter in the city’s intelligent evolution.
Carnegie Mellon University: American Robotics in Pittsburgh
Carnegie Mellon University has America’s largest robotics research center, setting it apart from other STEM universities such as Caltech, MIT, and Stanford. Professor Martial Hebert, Director of the Robotics Institute, tells us, “when the institute was founded there was a decision that it should have its own budgets, faculty, and researchers. This created a unique identity.”
In 1979 CMU secured US$5 million (roughly 32 million today) from Westinghouse Electric President Tom Murrin to fund the Robotics Institute. Nine years later the institute granted its first PhD in robotics.
In 1980 Professor Marc Raibert created CMU’s Leg Lab, which later gestated the famous spinoff company Boston Dynamics. Professor Red Whittaker developed a robot vehicle to help clean up the Three Mile Island nuclear meltdown in 1979. Professor Takeo Kanade’s virtualized reality system “EyeVision” broadcast 3-D replays of the 2001 NFL Super Bowl. And long before big tech companies and auto-manufacturers jumped in, NavLab introduced the first self-driving vehicle.
Today the Robotics Institute has 116 faculty members, 33 labs, and 98 ongoing projects under its umbrella.
To further commercialize research and development, the National Robotics Engineering Center (NREC) opened in 1996 with the support of NASA. The center receives both government and corporate contracts for agriculture, mining, nuclear, space, and defence projects. “Government contracts are long-term that run for five to ten years,” explains Professor Hebert, “while corporate contracts run for one to three years. We prefer programs that address ideas we can built on.”
Students are deeply involved in the process of spawning next-generation robots. In the basement of Cohon University Center, members of the Robotics Club tinker with drones, quadcopters, humanoid robots, and other such machines.
Brad Powell, a master student in electrical engineering, says Randy Pausch’s lecture “Really Achieving Your Childhood Dreams” inspired him to apply to Carnegie Mellon University. “It’s the creative freedom and integrity that attracted me here. Many people join the Robotics Club for the opportunity to work on research projects. I am currently working on the Lunar Rover project for Red Whittaker. There are other students working in the NavLab or computer vision lab.”
Students appreciate not only the top-level tutoring, but also the close relationships with their professors. As one student tells us, “We know [Associate Teaching Professor] David Kosbie will take time out of his day to coach students, not just on computer science, but also on being an adult. A friend of mine happened to walk with him across the lawn, and by the end of the walk had received a new workout regiment and advice on diet problems!”
Carnegie Mellon University: The Frontier of Machine Learning Research
“CoBot” greets visitors to CMU’s Gates Hillman Center. The robot can deliver messages, transport objects, and escort visitors. In a very human way, CoBot will ask passersby for help if it runs into a problem; and generate an excuse if it’s late. As an instrument to study real-time navigation and multi-robot multi-task planning, Cobot is constantly “learning” new capabilities.
Professor Manuela Veloso, Head of CMU’s Machine Learning Department, is the chief researcher behind CoBot. With an M.Sc. in electrical engineering from the Instituto Superior Técnico in Lisbon, Professor Veloso first joined CMU as a PhD student in the late 1980s, driven by her passion for industrial automation. When asked about the “techie” appearance of her robots, she dismisses the idea that robots should be built to resemble humans: “Robots should stay like robots. You don’t make a fridge look like a human. I care about whether they work autonomously, not how they look.”
This functional view of machine intelligence carries its genes from the school’s earliest founders. “Computer science research started at CMU in the early 1950s, when Herbert Simon and Allen Newell co-founded the Graduate School of Industrial Administration, conducting research on symbolic reasoning and computation,” says Professor Veloso. Newell went on to win the 1975 Turing award, and Simon was awarded the 1978 Nobel Prize in Economic Sciences. “I remember Allen Newell saying to us, ‘It’s easy to talk about what you would like a computer to do, but it’s hard to make them actually do it’.”
“You need to understand that artificial intelligence will not go away, it will be more and more present in our lives. A lot of the stuff we have today is called human-computer interaction (HCI). But I don’t call it HCI,” says Professor Veloso. “It’s human-AI interaction — we are interacting with artificial intelligence. And this is a new field of study.”
The Machine Learning Department is under the School of Computer Science, one of the seven CMU schools. It has 22 core faculty members, 40 faculty members, and approximately 60 PhD students.
In recent years there has been an upsurge of interest in deep learning from international applicants. Chenghui Zhou is one of six Chinese students out of a total of eight accepted into the machine learning PhD program last year.
Zhou’s thesis involves enabling a robot to guide the blind using deep reinforcement learning. She says the requirements proposed by her supervisor Professor Veloso are very specific, for example, “when the program sees a person greeting its blind owner, it will stop and remind the agent to wave back.” As a first-year student, she is still working on how to make the robot intercept a moving agent.
Zhou says she didn’t know what she signed up for when accepting her admission offer. “My dad was a computer science professor, and when I was a kid I always thought he was very idle. Being a PhD student really means working 24/7, but also feeling very unproductive at times. Actually my family is against girls doing a PhD. I think that is a myth, I think girls can do just as well.” Zhou may find inspiration in her supervisor Professor Veloso, who was keynote speaker at the 2015 Grace Hopper Celebration of Women in Computing conference, and has done much to encourage women in computing.
Former head of the department Professor Tom Mitchell has an office right across the hall from Zhou. His current research involves using statistical machine learning algorithms to analyze fMRI data, teaching mobile phones to learn from user instructions, and a software system called Never Ending Language Learning (NELL).
NELL is an endlessly inferencing machine that categorizes semantic knowledge on the web. It has been running nonstop since 2010, making it an open-ended approach to the limitations of the former expert system that failed.
There are eight different algorithms deployed on NELL, each helping to affirm the others’ observations. Explains Professor Mitchell, “One method is looking at the statistics of surrounding clauses, like when you say ‘mayor of Pittsburgh’, Pittsburgh is probably a city. The other method is looking at character sub-strings like “burgh”, which is a common suffix for cities. If algorithms make independent errors they can help each other correct them.”
One of NELL’s recent learned facts marked at 100.0 confidence is that Christopher Nolan directed the movie “Batman Begins.” This piece of information is stored in NELL’s knowledge repository to help the system become “wiser.”
NELL also fact-checks with a system called Never Ending Image Learner (NEIL) from Professor Abhinav Gupta’s research team, whereby the two systems exchange inferences on visual and semantic knowledge respectively. “NEIL is crawling, collecting, and classifying images. It is communicating with NELL, and they are teaching teach other,” says Mitchell. In the process, NELL and NEIL may help machines break free from the present constraints of labelled data.
University of Pittsburgh: AI Augmented “Eds and Meds”
Over the past few years, Professor Andrew Schwartz has been working on a neural prosthetic project that helps immobilized patients restore arm and hand functions. Based at the University of Pittsburgh’s Neurobiology Department, Professor Schwartz merges domain expertise with a group of 20 researchers including electric engineers, bioengineers, statisticians, and machine learning scientists.
“We have electrodes on the brain that send signal waves, while decoded signals are used to move prosthetic arms,” explains Professor Schwartz. “The system get two types of information streams, one from the camera’s visual artificial sensor, and the other from the brain. They are merged to make the robot work.” The robot arm is benefiting from machines’ increasing capability to analyze massive parallel streams of data.
Professor Schwartz believes that the present research on neural prosthetics, as pre-approved by the FDA, still requires two to three years of refinement: “We still want to remove cords from the skull for wireless transmission, and develop more dexterous arms with better materials.” Moreover, he admits we still don’t have a complete understanding of how the brain makes everything work. “Humans have 10 degrees of freedom on our arms and shoulders and 20 degrees of freedom in our hands. At this moment we still don’t know how brain signals get transmitted to the spinal cord and limbs.”
Supporting Professor Schwartz’s research is the University of Pittsburgh Medical Center (UPMC) — the largest healthcare and insurance provider in Pennsylvania, with 3.3 million members, 25 merged hospitals and 3,800 practicing physicians. The hospital’s innovation unit UPMC Enterprises shares an office with Google at Bakery Square, where there are currently 250 employees including data scientists and technologists.
UPMC was the birthplace of PACS, one of the earliest medical data archiving systems. The hospital receives petabytes worth of data, an amount that is doubling every 18 months. Keeping up with the data volume for real-time decision-making has become a priority.
A few years back, UPMC set up Covergence, a tablet-based platform which helps physicians access patient records. The project however failed to scale outside of UPMC, and was shut down. Dr. Shrestha explains that “at hospitals we don’t like to say ‘failing’, but at the innovation unit our approach is ‘agile’ and we let bad ideas fail fast.”
The innovation unit also invests in healthcare startups, one example being Vivifyhealth. “We monitor patient vitals with their consent. If the patient knows that they are falling sick, or that they are about to fall sick, we can prevent that episode and intervene.” says Dr. Shrestha. “Patients leave the hospital with [tracking] technology instead of pills.”
The current developments in artificial intelligence are in alignment with the hospital’s strategy. “There are many applications based on structured textual data including medical, allergies and lab values; and on unstructured data including surgical discharge summaries, radiology reports, and lab summaries. There are increasing capabilities in pixel data as well for image pattern recognition,” explains Dr. Shrestha.
UPMC has invested close to US$2 billion in technological innovation. In 2017 the hospital announced an R&D partnership with Microsoft, which involves the use of cloud and AI technology to help digitize its massive reservoir of paper documentation.
IAM Robotics: Founding a Warehouse Robotics Startup
After earning his PhD in Robotics, Mechanical Engineering and Electrical Engineering from the University of Florida, Tom Galluzzo came to Pittsburgh in 2009 to work at NREC as a robotics engineer. Four years later, he founded IAM Robotics, a company selling computer vision augmented robot arms for inventory retrieval.
IAM Robotics is situated in an industrial outskirt of Pittsburgh. The team includes engineers from CMU, NREC, University of Pittsburgh, and Pennsylvania State. For Galluzzo, a benefit of starting in Pittsburgh is access to 100 qualified CMU robotics graduates every year.
The company’s product robot called “Swift” uses an onboard RGB camera and gripper to retrieve items from a shelf. The gripper is capable of 1.5 lbs of suction power, which is adaptable for boxes and packaged goods. Swift’s computer vision system can see a wide breadth of items in different orientations, while the sensor’s depth component has its own infrared projectors projecting patterns on the scene. The robot is also connected to a scanner called “Flash,” with which a human helper scans new products into the system before assigning pick-up tasks to the robot.
Swift’s performance is equivalent to one full time worker, with one battery charge sustaining the machine for 10 hours. A business installing the system can expect to break even in two to three years.
IAM Robotics’ present customers include Rochester Drug Cooperative, one of the largest healthcare distributors in the US. Galluzzo sees many opportunities for Swift in E-commerce logistics demanding pick and pack services: “People in the US spend 40 billion hours shopping for consumer goods. To replace that, it would take more than the entire US unemployment pool. Without physical automation, there’s no way to scale pick, pack and ship.”
Like most American industrial robot companies, IAM Robotics outsources compartment design, and does in-house assembly of FANUC robot arms and sensors.
Galluzzo says a big hurdle in Pittsburgh is funding. “Without doubt, that took a long time. We had seed funding from Innovation Works, but it’s still challenging.” Galluzzo told Synch that IAM Robotics is hoping to announce good news soon.
Swartz Center for CMU based Ecosystem
The Swartz Center for Entrepreneurship plays a central role in leveraging Pittsburgh’s AI startups and spinoffs. It was founded in 2015 with a US$31 million gift from alumnus and venture capitalist James Swartz to support CMU entrepreneurial activity.
According to a report by Ernst & Young and Innovation Works, “from 2012 to 2016, 318 unique companies attracted funding totalling roughly US$1.7 billion in Pittsburgh”. The most-invested categories were software, life sciences (biotech, medical devices, healthcare IT and health care services), followed by hardware (robotics and electronics).
Six years ago, former serial entrepreneur and venture capitalist Dave Mawhinney was hired by the centre to revive the local startup scene. Mawhinney’s office is located on the second floor of Tepper Business School. He recently flew back from Silicon Valley, and is meeting with investors from Taiwan.
Mawhinney says many international investors are knocking on the door of Pittsburgh’s innovation scene, with particular interest from China. “There has been a lot of tire kicking, and we’re starting to see potential. We are working with the firm Sinovation on several potential deals. Chinese money is funding companies in the US now.”
Mawhinney is however concerned about Pittsburgh’s lack of early risk venture capital from domestic investors, and local companies’ reluctance to deal with startups. “It’s different in Silicon Valley, where today’s startup will be a mature company in ten years, and so people are willing to take the risk.”
AI, IP, and an Innovative Economy
In 2016, the city’s three biggest universities produced 145 patents, up 43% from 2012. Intellectual property licensing is one of the business niches for CMU’s NREC, which currently holds 659 distinct patents.
In a recent high-profile case, CMU sued semiconductor giant Marvell for IP infringement. The case settled for US$750 million last year, the second-highest amount in patent compensation history. Proceeds went to patent holder Professor José Moura and his former student Aleksandar Kavcic. A year later the duo, along with Professor Moura’s wife Professor Veloso, donated US$16.5 million to CMU for data science and engineering research.
The Center for Technology Transfer and Enterprise Creation (CCTEC) was created to walk startups through the legal process, and help them navigate intellectual property issues.
In contrast to the bleak 1980s, Pittsburgh’s campuses today are vibrant, its robots are robust and the city has lost its patina of rust. An intelligent engine is driving the transformation by attracting new people, new ideas, and new energy.
This is the third article within featured series AI and Globalization, click to read Building AI Superclusters in Canada, Canvassing Switzerland’s AI Landscape
Journalist: Meghan Han | Editor: Michael Sarazen | Producer: Chain Zhang
Pingback: Synced Review | Canvassing Switzerland’s AI Landscape
Pingback: Synced Review | Building AI Superclusters in Canada