Open Access

ARTICLE

Unsupervised Semantic Segmentation Method of User Interface Component of Games

Shinjin Kang¹, Jongin Choi^2,*

1 School of Games, Hongik University, Sejong, 30016, Korea
2 Department of Digital Media, Seoul Women’s University, Seoul, 01797, Korea

* Corresponding Author: Jongin Choi. Email:

Intelligent Automation & Soft Computing 2022, 31(2), 1089-1105. https://doi.org/10.32604/iasc.2022.019979

Received 04 May 2021; Accepted 02 July 2021; Issue published 22 September 2021

Abstract

The game user interface (UI) provides a large volume of information necessary to analyze the game screen. The availability of such information can be functional in vision-based machine learning algorithms. With this, there will be an enhancement in the application power of vision deep learning neural networks. Therefore, this paper proposes a game UI segmentation technique based on unsupervised learning. We developed synthetic labeling created on the game engine, image-to-image translation and segmented UI components in the game. The network learned in this manner can segment the target UI area in the target game regardless of the location of the corresponding component. The proposed method can help interpret game screens without applying data augmentation. Also, as this scheme is an unsupervised technique, it has the advantage of not requiring paring data. Our methodology can help researchers who need to extract semantic information from game image data. It can also be used for UI prototyping in the game industry.

---

Keywords

---