IQM Games

Automatic Detection of Game Engine Artifacts Using Full Reference Image Quality Metrics

Rafał Piórkowski                             Radosław Mantiuk                            Adam Siekawa
West Pomeranian University of Technology, Szczecin

Published in:
Journal
ACM Transactions on Applied Perception (TAP)
Volume 14 Issue 3, March 2017
Article No. 14
ISSN: 1544-3558

The project was partially funded by the Polish National Science Center (grant number DEC-2013/09/B/ST6/02270).

Abstract:
Contemporary game engines offer an outstanding graphics quality but they are not free from visual artifacts. A typical example is aliasing, which, despite advanced antialiasing techniques, is still visible to the game players. Essential deteriorations are the shadow acne and peter panning responsible for deficiency of the shadow mapping technique. Also Z-fighting, caused by the incorrect order of drawing polygons, significantly affects the quality of the graphics and makes the gameplay more difficult. In this work, we propose a technique, in which visibility of deteriorations is uncovered by the objective image quality metrics (IQMs). We test the efficiency of a simple mathematically based metric and advanced IQMs: a Spatial extension of CIELAB (S-CIELAB), the Structural SIMilarity Index (SSIM), the Multiscale Structural SIMilarity Index (MS-SSIM), and the High Dynamic Range Visual Difference Predictor-2 (HDR-VDP-2). Additionally, we evaluate the Color Image Difference (CID) metric, which is recommended to detect the differences in colors. To find out which metric is the most effective for the detection of the game engine artifacts, we build a database of manually marked images with representative set of artifacts. We conduct subjective experiments in which people manually mark the visible local artifacts in the screenshots from the games. Then the detection maps averaged over a number of observers are compared with results generated by IQMs. The obtained results show that SSIM and MS-SSIM metrics outperform other techniques. However, the results are not indisputable, because, for small and scattered aliasing artifacts, HDR-VDP-2 metrics report the results most consistent with the average human observer. As a proof of concept, we propose an application in which resolution of the shadow maps is controlled by the SSIM metric to avoid perceptually visible aliasing artifacts on the shadow edges.


Screenshot from the cunducted perceptual experiment

Video preview

Downloads:
Supplementary materials
DOI: 10.1145/3047407

Citation:

@article{piorkowski2017automatic,
  title={Automatic Detection of Game Engine Artifacts Using Full Reference Image Quality Metrics},
  author={Pi{\'o}rkowski, Rafa{\l} and Mantiuk, Rados{\l}aw and Siekawa, Adam},
  journal={ACM Transactions on Applied Perception (TAP)},
  volume={14},
  number={3},
  pages={14},
  year={2017},
  publisher={ACM}
}

References:
Rafał Piórkowski, Radosław Mantiuk. Calibration of Structural Similarity Index Metric to Detect Artefacts in Game Engines. Lecture Notes in Computer Science (Proc. of ICCVG’16), 2016, Vol. 9972, pp. 86-94.

@Inbook{Piorkowski2016,
  author="Pi{\'o}rkowski, Rafa{\l} and Mantiuk, Rados{\l}aw",
  editor="Chmielewski, Leszek J. and Datta, Amitava and Kozera, Ryszard and Wojciechowski, Konrad",
  title="Calibration of Structural Similarity Index Metric to Detect Artefacts in Game Engines",
  bookTitle="Computer Vision and Graphics: International Conference, ICCVG 2016, Warsaw, Poland, September 19-21, 2016, Proceedings",
  year="2016",
  publisher="Springer International Publishing",
  pages="86--94",
  isbn="978-3-319-46418-3",
  doi="10.1007/978-3-319-46418-3_8",
  url="http://dx.doi.org/10.1007/978-3-319-46418-3_8" 
}
Rafał Piórkowski, Radosław Mantiuk. Using full reference image quality metrics to detect game engine artefacts. Proceedings of the ACM SIGGRAPH Symposium on Applied Perception, 2015, pp. 83-90.

@inproceedings{piorkowski2015using,
  title={Using full reference image quality metrics to detect game engine artefacts},
  author={Pi{\'o}rkowski, Rafa{\l} and Mantiuk, Rados{\l}aw},
  booktitle={Proceedings of the ACM SIGGRAPH Symposium on Applied Perception},
  pages={83--90},
  year={2015},
  organization={ACM}
}