SJTU’s MedMNIST Classification Decathlon: Lightweight AutoML Benchmarking for Medical Image Analysis

If there is one domain where professionals and the public are positively united on AI’s promise and potential, it may be medical image analysis, where SOTA computer vision (CV) techniques and image classification datasets are already outperforming human radiologists on tasks such as tumour and cancer detection. Now, a team of researchers from Shanghai Jiao Tong University (SJTU) has built a benchmark suite designed to help make these life-saving models even stronger.

Medical image analysis is the science of analyzing and processing biomedical images such as X-rays, CT scans and ultrasounds. While AutoML approaches continue to improve, the researchers note that there are currently few effective benchmarks for comparing these systems. To address the issue, they developed MedMNIST, a collection of 10 pre-processed medical image datasets released under Creative Commons (CC) licenses.

SJTU’s MedMNIST datasets are systematized to perform classification tasks on lightweight 28×28 images, which requires no clinical or CV background knowledge. They cover primary data modalities in medical image analysis and include diverse data scales (from 100 to 100,000).

The team designed the MedMNSIT Classification Decathlon to function as a benchmark for AutoML in medical image classification that can effectively demonstrate the performance of AutoML algorithms on all ten MedMNIST datasets without manual tuning.

The researchers say the datasets perform well in general with the Google AutoML Vision Method. They note however that typical statistical machine learning algorithms such as ResNet-18, ResNet-50, and autoKeras did not work well on the MedMNIST datasets.

The team says it hopes the MedMNIST Classification Decathlon benchmarks will help researchers speed up their medical image analysis algorithm studies.

The paper MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis is on arXiv. The MedMNIST datasets and code are on the project GitHub.

---

Analyst: Robert Tian | Editor: Michael Sarazen; Fangyu Cai

---

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle. Along with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

---

Thinking of contributing to Synced Review? Synced’s new column Share My Research welcomes scholars to share their own research breakthroughs with global AI enthusiasts.