Snow White’s evil stepmother utters one of animation’s classic lines: “Mirror, mirror, on the wall, who’s the fairest of them all?” If the old hag were an artificial intelligence, she’d probably spend the entire film just attempting to locate that mirror — which, as it happens, is a relatively difficult computer vision challenge.
A group of Chinese researchers have come up with a novel method for identifying mirrors in images which outperforms state-of-the-art detection and segmentation methods on targeted baselines.
Mirrors reflect content in their own environment, and that continuity makes recognizing mirrors a formidable task even for cutting-edge computer vision systems. The researchers note that no previous study has put a specific focus on the mirror segmentation problem, and that theirs is the first automatic method developed for the task.
Because model training can’t proceed without data, researchers first built a mirror dataset that includes 4,018 pairs of images (original image and segmentation image) containing mirrors and manually annotated masks.
The proposed “MirrorNet” network is a cascading framework that contains three modules:
- a pre-trained Feature Extraction Network (ResNeXt101 network) that extracts multi-scale feature maps from input images;
- Contextual Contrasted Feature Extraction (CCFE) modules connected to a Feature Extraction Network, which learn different scales of contextual contrasted features for localizing mirrors of different sizes;
- a mirror map that coarsely highlights the dividing boundaries of the mirror and progressively refines itself by helping the upper CCFE layers focus on learning finer contextual contrasted features.
In their experiments, researchers adopted five evaluation metrics commonly used in similarly detection and segmentation tasks, and compared MirrorNet with 11 state-of-the-art methods such as PSPNet, ICNet and Mask RCNN. MirrorNet achieved the best performance, with a large margin over the other methods.
Researchers focused their study on interior mirrors and did not include outdoor mirrors such as the reflective glass walls of skyscrapers or large mirrors in public places or outside shops, etc. They suggest that extending MirrorNet to a wider range of scenarios in the future may benefit for example autonomous driving and drone navigation research.
The authors are from the Dalian University of Technology, Peng Cheng Laboratory, and City University of Hong Kong. The paper Where Is My Mirror? has been accepted by ICCV 2019 and is on arXiv.
Journalist: Tony Peng | Editor: Michael Sarazen