Chinese Opera Character Painting Style Transfer: Using AI to Generate and Preserve Art

Chinese Opera Character Painting Style Transfer: Using AI to Generate and Preserve Art

Xirong Gao^1*, Liza Mohammad Noh², Zhongwei Huang³

 ^1,2College of Creative Arts, University Teknologi MARA (UiTM), Shah Alam, Selangor Darul Ehsan, Malaysia

³School of Computer Science and Engineering, Macau University of Science and Technology

Taipa, Macau, China

*Corresponding author

DOI: https://dx.doi.org/10.47772/IJRISS.2024.803428S

Received: 20 November 2024; Accepted: 23 November 2024; Published: 23 December 2024

ABSTRACT

Chinese Opera characters ink painting, a distinctive blend of Chinese color ink painting and traditional opera, reflects the rich aesthetic heritage of Chinese culture. The advent of Artificial Intelligence Generated Content (AIGC) technology presents new opportunities for preserving and innovating this traditional art form. While style transfer techniques have been widely applied to Western art, the freehand style of Chinese ink painting remains under-explored. This paper fills this gap by constructing datasets of Chinese Opera character paintings through field visits and web crawling. This paper develops an automated system for transforming realistic opera character images into Chinese opera character paintings by leveraging generative adversarial networks (GAN) technology. The generated results prove that the GAN-based model is able to learn the key features of a style image and be able to distinguish the relationship between people in the image, it is better than an ordinary person with no foundation would draw. This research advances AI’s application in traditional art and provides a new thought to the preservation, dissemination, and modern reinterpretation of Chinese Opera characters ink painting.

Keywords: Chinese Opera Characters Ink Painting, Cultural Heritage, Style Transfer, Artificial Intelligence, Generative Adversarial Network

INTRODUCTION

Chinese ink painting is an artistic and cultural treasure (Wang et al., 2023). The Chinese Opera characters ink painting is a combination of Chinese color ink painting and traditional opera (Lemenkova, 2017). It has gone through a long evolution, it formed in Tang Dynasty and matured in Qing Dynasty. Until now, it has undergone a transformation from classical to modern, but the special theme of Chinese Opera characters in ink painting has not changed too much. Specifically, the Chinese Opera characters ink painting focuses on the smoothness and dynamic expression of lines and pursues the vividness and harmonious combination of colors. After thousands of years of development, Chinese traditional culture has formed its style, its techniques, and a complete artistic system, reflecting the aesthetic taste of the Chinese nation.

Figure 1 shows the famous opera character paintings of “Farewell My Concubine” (Yu, 2024). Opera character paintings can be interpreted in two ways, the first is the direct reproductions of stage characters, and it also blends personal emotions with their understanding of the opera. However, Chinese Opera characters’ ink painting requires one to be extremely professional and artistic, mastering the painting skills of Chinese art and understanding the nuances of opera are significant barriers for average people.

Figure 1. The Chinese Opera Character Paintings

(Source:Author’s collection )

In recent years, the rapid development of artificial intelligence technology, especially Artificial Intelligence Generated Content (AIGC) technology enables text generation, image generation, and even video generation to become a reality (Wu et al., 2023). Inspired by this, the automated creation of Chinese ink painting is crucial to the preservation and inheritance of this art form. As shown in Figure 2, the style transfer can convert the real picture of any landscape (i.e., content image) into Van Gogh’s style painting based on the given style image (Gatys et al., 2016). Previous research such as (Liu, 2021) and (Galerne, 2024) have focused primarily on style transfer techniques for Western paintings. In contrast, the more freehand style of Chinese ink painting has not been well studied.

To fill this gap, firstly, build a datasets of Chinese Opera Character Paintings through field visits and web crawling, then employ artificial intelligence technology generative adversarial networks (GAN) to learn the artistic characteristics and convert the opera characters pictures into Chinese opera character paintings based on collected realistic paintings.

Figure 2. The Style Transfer of Landscape Image.

(Source: https://www.subsubroutine.com/sub-subroutine/2016/11/12/painting-like-van-gogh-with-convolutional-neural-networks)

LITERATURE REVIEW

The Style Transfer of Western Painting

As of now, a lot of research has studied the image and painting style transfer, but they are mainly focused on Western painting.

Gatys et al. (2016) conducted a pioneered work toward convolutional neural network (CNN) based Western painting style transfer. However, since the network needs to be trained for each transfer, the speed is very slow, and real-time transfer cannot be achieved. Many works and techniques have been proposed to enhance the performance of image style transfer. For example, Johnson et al. (2017) proposed a faster approximation style transfer method using feed-forward neural networks, to address the slow transfer speed. Huang and Belongie (2017) achieved real-time transfer of arbitrary styles using an Adaptive Instance Normalization layer (AdaIN), this method aligns the mean and variance of content features with the mean and variance of style features, it achieves comparable speed and is not restricted to a predefined set of styles. Zhu et al. (2017) incorporate cycle-consistency loss in image-to-image translation, this ensures that when translating from one domain to another and then back to the original domain, the final image should be as similar as possible to the original image. This cycle consistency requirement enables the model to learn the mapping between two different domains without paired training data. Liu (2021) propose an improved GAN-based on gradient penalty, to realize the image oil painting style transfer and reconstruction. Galerne et al. (2024) proposed a solution for ultra-high resolution (UHR) images transfer to painting, enabling multi-scale neural style transfer (NST) at unprecedented image sizes.

The Style Transfer of Chinese Painting

The style transfer of Chinese style paintings appeared more later, recently, a few researchers have brought the style transfer of traditional Chinese paintings to the public’s attention.

Li et al. (2019) present a neural abstract style transfer method for Chinese traditional painting. Sheng et al. (2019) convert natural landscape images into Chinese paintings based on the CNN. Peng et al. (2022) proposed a method to achieve image style transfer from landscape photos to Chinese landscape paintings based on CycleGAN. Zhang et al. (2022) proposed a style transfer method for Peking Opera makeup, and Zhang et al. (2023) designed a ChinaOperaGAN framework suitable for opera makeup style transfer. Wang et al. (2023) proposed an asymmetric cyclic consistency GAN for Chinese ink painting style transfer, and they build a Chinese bird ink painting dataset to validate the effectiveness of the model. Yan et al. (2023) used a Style Generative Adversarial Network to learn implicit features of Peking Opera facial masks and generate new facial masks, however, this research focuses only on facial features.

The style transfer of Chinese opera character color ink painting has not been well studied. Although existing work has involved the style transfer of traditional Peking Opera faces, it is only a partial facial style transfer, completely different from the style transfer of opera character color ink paintings that this article aims to solve. This article aims to solve a more complicated problem that includes not only facial features but also clothing, decorations, props, etc.

RESEARCH METHODOLOGY

Research Overview

This article adopts the observation method and qualitative analysis. A large number of Chinese opera character ink painting datasets were first collected through web crawlers and field visits and then used artificial intelligence technology Generative Adversarial Network (GAN) (Goodfellow et al. 2020) for style transfer training. Analyze the performance of style transfer results by observing and comparing the results obtained with different experimental settings and style images.

Generative Adversarial Network Design

The overall network structure of the proposed style transfer model is presented in Figure 3, The model utilizes the CycleGAN network structure, comprising two pairs of generators and discriminators, along with adversarial networks for training. The system architecture includes the input layers, the feature map process layer, the encoding layer, the decoding layer with the AdaLIN, and the output layer.

Figure 3. The Workflow of Proposed GAN-based Style Transfer Model

(Source:Author’s collection )

The overall workflow is presented in Figure 3, first, before the generator processes the input, the model fuses the real image with its feature map to minimize the network’s focus on blank areas. Second, during the down-sampling phase in the encoder, this study employs the Convolutional Block Attention Module (CBAM) (Woo et al., 2018) to guide the network in emphasizing critical features. Additionally, the adaptive normalization function layer is incorporated into the generator’s decoder to enhance the stylization of the image.

The discriminator has four convolutional layers and a resnet block, the discriminator determines whether the currently generated color ink painting of opera characters is real or fake. When the paintings generated by the generator can deceive the discriminator, the quality of the generated picture will meet the requirements.

Then using the attention module CBAM focuses on specific areas of the image to help the network transform the key areas. The adaptive normalization operations enhance the stability of neural networks, allowing them to converge faster and exhibit stronger generalization capabilities. They also improve the model’s feature learning ability and overall performance.

Data Collection

Due to the lack of relevant datasets, this research collected a substantial amount of both paired and unpaired data through field visits and web scraping. The source domain consists of 108 real Chinese opera character photos, while the target domain includes 984 Chinese opera character paintings. Each real Chinese opera character photo in this dataset corresponds to 5-10 character paintings. The ratio of training set and test set is set to 8:2.

The experiments in this paper are run on the Google Colab platform based on Python3 and Pytorch, and the code is available online[1].

Results Analysis

The input of the GAN model is the content image, and then the model can convert the original image into a Chinese color ink painting with the same style according to the provided style image, as shown in Figure 4, selected three typical Chinese opera stage photos, including (a) The Yong’s Female Warrior, (b) Farewell My Concubine and (c) The Universal Pagoda. The output results are shown in the last column. It is obvious that the model has learned the typical features of the style image and can distinguish the boundaries of the characters in the content image very well, assigning the elements of the style image to the corresponding positions in the content image.

In general, sample (a) has the best generation performance, sample (b) has a very realistic and vivid transfer effect, but it does not distinguish the stage background very well, and (c) has a generation effect that is almost consistent with the style picture, reaching a level that is indistinguishable to the naked eye, but it also does not distinguish the stage background very well.

Figure 4. The Style Transfer Performance of Different Content images.

(Source:Author’s collection )

The subsequent experiments verified the generation effect of the same content image under different style images. This study used two content images, each has two style images. The results are shown in Figure 5. The model can indeed adjust the style of the generated images according to different style images, and surprisingly, the model can distinguish the different characters in the picture. For example, in Figure 5 (a), the male character in the style image wears red clothes, while the female character wears white and blue clothes. In the corresponding generated images, the model assigns corresponding colors to different characters, although the positions of the characters in the content image and the style image have changed. This can also be seen in the results of the second style image.

Compared with Figure 5 (a), the number of characters in the given style image in Figure 5 (b) is inconsistent with the number of characters in the content image. Therefore, the two different characters are given the characteristics based on the given style image, but the two characters also retain the differences in the content picture, such as the headdress of the left character is brighter in color. Compared to Figure 5(a), the generated image in Figure 5(b) is also less distorted. According to the output results, the transferred images have achieved sufficient vividness and detail requirements, so the transfer results are acceptable.

Figure 5. The Style Transfer Performance Comparison of Same Content Image but Different Style Image.

(Source:Author’s collection )

CONCLUSION

This paper studies the the style transfer of Chinese opera characters paintings, which is an earlier attempt that utilized AI technologies to Chinese traditional art generation. The results prove that the adopted GAN-based style transfer has an acceptable transfer performance, the generated painting is much better than an ordinary person with no foundation would draw. Surprisingly, the AI model can even distinguish between different characters in the image and assign corresponding features to them. The proposed method can be used to generate and inherit Chinese opera character color ink paintings.

The style transfer of Chinese opera character ink painting has practical applications in education and artistic fields. For example, in learning traditional techniques, style transfer technology can be used to digitally simulate traditional Chinese opera ink painting techniques, enabling students to learn and understand brushwork, composition, and color application without the need for expensive materials. By integrating this style with other subjects such as history, literature, and performing arts, cross-disciplinary education can teach the cultural and historical significance of Chinese opera and its artistic expressions. Bridging cultural gaps, educational programs in international settings can use style transfer to make Chinese opera art more accessible to global audiences, helping to foster understanding and appreciation for Chinese culture. Interactive displays, museums, and galleries can utilize style transfer to create interactive installations where visitors can transform their photographs or drawings into Chinese opera ink styles, enriching the exhibition experience.

However, some artifacts still exist in the generated paintings, such as the current method can not distinguish the boundary of the character well, and there are certain artifacts in the generated images. Future work may consider combining structure loss and color loss further to improve the realism and color vividness of generated images.

REFERENCES

1. Galerne, B., Raad, L., Lezama, J., & Morel, J. M. (2024, July). Scaling painting style transfer. Computer Graphics Forum, 43(4), Article e15155. https://doi.org/10.1111/cgf.15155
2. Gatys, L. A., Ecker, A. S., & Bethge, M. (2016). Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2414–2423. https://doi.org/10.1109/CVPR.2016.265
3. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139–144. https://doi.org/10.1145/3422622
4. Huang, X., & Belongie, S. (2017). Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 1501–1510. https://doi.org/10.1109/ICCV.2017.167
5. Johnson, J., Alahi, A., & Fei-Fei, L. (2017). Perceptual losses for real-time style transfer and super-resolution. In Proceedings of Computer Vision–ECCV 2016: 14th European Conference (Vol. 14, pp. 694–711). Springer. https://doi.org/10.1007/978-3-319-46475-6_43
6. Lemenkova, P. (2017, December). Treasures of Chinese culture: Painting and opera. In Introduction to Chinese Culture, December 2017, Qingdao, China. https://doi.org/10.13140/RG.2.2.14849.40808
7. Li, B., Xiong, C., Wu, T., Zhou, Y., Zhang, L., & Chu, R. (2019). Neural abstract style transfer for Chinese traditional painting. In C. Jawahar, H. Li, G. Mori, & K. Schindler (Eds.), Computer Vision – ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Part II (Lecture Notes in Computer Science, Vol. 11363, pp. 212–227). Springer. https://doi.org/10.1007/978-3-030-20890-5_14
8. Liu, Y. (2021). Improved generative adversarial network and its application in image oil painting style transfer. Image and Vision Computing, 105, 104087. https://doi.org/10.1016/j.imavis.2020.104087
9. Peng, X., Peng, S., Hu, Q., Peng, J., Wang, J., Liu, X., & Fan, J. (2022). Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings. Neural computing and applications, 34(20), 18075-18096. https://doi.org/10.1007/s00521-022-07432-w
10. Sheng, J., Song, C., Wang, J., & Han, Y. (2019). Convolutional neural network style transfer towards Chinese paintings. IEEE Access, 7, 163719–163728. https://doi.org/10.1109/ACCESS.2019.2952616
11. Wang, W., Li, Y., Ye, H., Ye, F., & Xu, X. (2023). Ink painting style transfer using asymmetric cycle-consistent GAN. Engineering Applications of Artificial Intelligence, 126, 107067. https://doi.org/10.1016/j.engappai.2023.107067
12. Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19). https://doi.org/10.1007/978-3-030-01234-2_1
13. Wu, J., Gan, W., Chen, Z., Wan, S., & Lin, H. (2023). AI-generated content (AIGC): A survey. arXiv preprint arXiv:2304.06632, https://doi.org/10.48550/arXiv.2304.06632.
14. Yan, M., Xiong, R., Shen, Y., Jin, C., & Wang, Y. (2023). Intelligent generation of Peking opera facial masks with deep learning frameworks. Heritage Science, 11(1), 20. https://doi.org/10.1186/s40494-023-00865-z
15. Yu, J. (2024). Nationalism and Identity Crisis: Analyzing “Farewell to My Concubine” Through Historical Contexts. Arts, Culture and Language, 1(7). https://doi.org/10.61173/b7360d62
16. Zhang, F., Cao, D., Ma, X., Chen, B., & Zhang, J. (2023). Style transfer network for generating opera makeup details. Journal of System Simulation, 35(9), 2064–2076. https://doi.org/10.16182/j.issn1004731x.joss.22-1499
17. Zhang, F., Liang, X., Sun, Y., Lin, M., Xiang, J., & Zhao, H. (2022). POFMakeup: A style transfer method for Peking Opera makeup. Computers and Electrical Engineering, 104, 108459. https://doi.org/10.1016/j.compeleceng.2022.108459
18. Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2223–2232). https://doi.org/10.1109/ICCV.2017.238

FOOTNOTES

[1] https://github.com/ZekaiShao25/cyclegan-for-style-transfer