Open Access

ARTICLE

Explicitly Color-Inspired Neural Style Transfer Using Patchified AdaIN

by Bumsoo Kim¹, Wonseop Shin², Yonghoon Jung¹, Youngsup Park³, Sanghyun Seo^1,4,*

1 Department of Applied Art and Technology, Chung-Ang University, Anseong, 17546, Republic of Korea
2 Department of Advanced Imaging Science Multimedia & Film, Chung-Ang University, Seoul, 06974, Republic of Korea
3 Innosimulation Co., Ltd., Gangseo-gu, 07794, Republic of Korea
4 School of Art and Technology, Chung-Ang University, Anseong, 17546, Republic of Korea

* Corresponding Author: Sanghyun Seo. Email:

Computer Modeling in Engineering & Sciences 2024, 141(3), 2143-2164. https://doi.org/10.32604/cmes.2024.056079

Received 13 July 2024; Accepted 25 September 2024; Issue published 31 October 2024

Abstract

Arbitrary style transfer aims to perceptually reflect the style of a reference image in artistic creations with visual aesthetics. Traditional style transfer models, particularly those using adaptive instance normalization (AdaIN) layer, rely on global statistics, which often fail to capture the spatially local color distribution, leading to outputs that lack variation despite geometric transformations. To address this, we introduce Patchified AdaIN, a color-inspired style transfer method that applies AdaIN to localized patches, utilizing local statistics to capture the spatial color distribution of the reference image. This approach enables enhanced color awareness in style transfer, adapting dynamically to geometric transformations by leveraging local image statistics. Since Patchified AdaIN builds on AdaIN, it integrates seamlessly into existing frameworks without the need for additional training, allowing users to control the output quality through adjustable blending parameters. Our comprehensive experiments demonstrate that Patchified AdaIN can reflect geometric transformations (e.g., translation, rotation, flipping) of images for style transfer, thereby achieving superior results compared to state-of-the-art methods. Additional experiments show the compatibility of Patchified AdaIN for integration into existing networks to enable spatial color-aware arbitrary style transfer by replacing the conventional AdaIN layer with the Patchified AdaIN layer.

---

Keywords

---