What Can Style Transfer and Paintings Do For Model Robustness?

Hubert Lin, Mitchell van Zuijlen, Sylvia C. Pont, Maarten W.A. Wijntjes, Kavita Bala

In CVPR 2021

[Paper] [Poster] [Slides] [Github] [Dataset]

TL;DR / TL;DW
1. Style transfer improves model robustness, but not necessarily through real painting styles.
2. Real paintings and stylized images capture complementary invariances.
 a. Real paintings improve robustness to corruptions and novel viewpoints.
 b. Stylization greatly improves robustness to corruptions but harms novel viewpoint generalization.
3. Real paintings improve robustness cost-effectively.
4. Other art forms, such as sketches, cartoons, and untextured renderings are unable to improve model robustness to the same extent as paintings.

@article{lin2021robustness,
title={What Can Style Transfer and Paintings Do For Model Robustness?},
author={Lin, Hubert and van Zuijlen, Mitchell and Pont, Sylvia C and Wijntjes, Maarten WA and Bala, Kavita},
journal={CVPR2021},
year={2021}
}