It’s All in the Eyes: Google AI Calculates Cardiovascular Risk From Retinal Images

A retinal fundus image is a photograph of the back of the eye taken through the pupil. For more than 100 years these images have been used for detecting eye disease. Now Google has introduced a surprising new use for retinal images: combined with artificial intelligence, they can also predict a patient’s risk of heart attack or stoke.

Research arm Google Brain today published a paper in the journal Nature Biomedical Engineering which demonstrates how deep learning models can use retinal images to detect a patient’s age, gender, smoking status and systolic blood pressure; calculate cardiovascular risk factors; and predict the risk of major adverse cardiac events occurring over the next five years.

A problem with today’s mainstream cardiovascular risk calculators such as the Pooled Cohort Equations, Framingham, and Systematic Coronary Risk Evaluation is that they require the input of multiple features such as blood pressure, body mass index, glucose and cholesterol levels, etc. to generate a disease risk result. A study by the American College of Cardiology’s Practice Innovation And Clinical Excellence Program concluded that the data required to calculate 10-year risk was available for less than 30% of patients.

Google Brain discovered that a retinal fundus image alone was sufficient to predict many cardiovascular risk factors. The anatomical feature patterns were extracted using a convolutional neural network — a computational model that excels in analyzing images.

Researchers trained models on retinal images from 284,335 patients and validated on two independent datasets of 12,026 and 999 patients. The trained model identified patients’ ages with 3.26 years, distinguished gender 97 percent of the time, spotted a smoker 71 percent of the time, and calculated blood pressure with a 11.23 mmHg margin of error.

Google Brain then took a step forward. Researchers discovered the trained model could predict a patient’s risk of cardiovascular disease over the next five years 70 percent of the time, approaching the accuracy rate of established risk calculators without all the additional data inputs.

Deep learning is often criticized for its lack of transparency and interpretability, and this has hindered the technology’s entry into areas such as medical health and the legal system. But Google Brain believes their methodology is sound. It employs attention techniques to determine which pixels are the most important for predicting a specific cardiovascular risk factor: blood vessels for example are a critical feature for determining blood pressure.

This is not the first time Google Brain has leveraged the value of retinal images. In November 2016 it presented a study on deep learning for early detection of diabetic retinopathy, which could potentially protect 415 million worldwide diabetics from irreversible blindness.

Google Brain’s paper opens up the exciting possibility of applying deep learning to retinal images for improving diagnoses beyond eye disease. Will AI be the key that unlocks even more medical science innovations?