Open Access

ARTICLE

PP-GAN: Style Transfer from Korean Portraits to ID Photos Using Landmark Extractor with GAN

Jongwook Si¹, Sungyoung Kim^2,*

1 Department of Computer AI Convergence Engineering, Kumoh National Institute of Technology, Gumi, 39177, Korea
2 Department of Computer Engineering, Kumoh National Institute of Technology, Gumi, 39177, Korea

* Corresponding Author: Sungyoung Kim. Email:

(This article belongs to the Special Issue: Advanced Artificial Intelligence and Machine Learning Frameworks for Signal and Image Processing Applications)

Computers, Materials & Continua 2023, 77(3), 3119-3138. https://doi.org/10.32604/cmc.2023.043797

Received 12 July 2023; Accepted 27 October 2023; Issue published 26 December 2023

Abstract

The objective of style transfer is to maintain the content of an image while transferring the style of another image. However, conventional methods face challenges in preserving facial features, especially in Korean portraits where elements like the “Gat” (a traditional Korean hat) are prevalent. This paper proposes a deep learning network designed to perform style transfer that includes the “Gat” while preserving the identity of the face. Unlike traditional style transfer techniques, the proposed method aims to preserve the texture, attire, and the “Gat” in the style image by employing image sharpening and face landmark, with the GAN. The color, texture, and intensity were extracted differently based on the characteristics of each block and layer of the pre-trained VGG-16, and only the necessary elements during training were preserved using a facial landmark mask. The head area was presented using the eyebrow area to transfer the “Gat”. Furthermore, the identity of the face was retained, and style correlation was considered based on the Gram matrix. To evaluate performance, we introduced a metric using PSNR and SSIM, with an emphasis on median values through new weightings for style transfer in Korean portraits. Additionally, we have conducted a survey that evaluated the content, style, and naturalness of the transferred results, and based on the assessment, we can confidently conclude that our method to maintain the integrity of content surpasses the previous research. Our approach, enriched by landmarks preservation and diverse loss functions, including those related to “Gat”, outperformed previous researches in facial identity preservation.

---

Keywords

---