This project leverages a curated dataset from Kaggle's "Best Artworks of All Time," encompassing 50 most influential artists and 17,548 distinct images.
Conduct image classification to accurately categorize paintings of different artists.
- Constructed a custom CNN and performed extensive hyperparameter tuning to optimize its performance.
- Constructed a Transfer learning Model based on the ResNet50 architecture, capitalizing on pre-trained weights to boost accuracy.
Conduct style transfer to mimic and transfer artistic styles.
- Implemented style transfer models utilizing both VGG19 and ResNet50 architectures. The goal was to capture the artistic essence of one painting and project it onto another.
- To delve deeper into artistic transformations, CycleGAN was employed. It was trained using a diverse dataset comprising natural landscapes and architectural imagery. The overarching ambition was to metamorphose these images to resonate with Claude Monet's iconic painting style.
To bridge the gap between advanced computational models and end-users, we launched three web interfaces with Streamlit:
- An interface for classification based on the model we trained (custom CNN and transfer learning based on ResNet50)
- An interface for the VGG19 and ResNet50-based style transfer.
- An interface for the Monet-inspired CycleGAN style transformations.
- Main Dataset: Best Artworks of All Time (https://www.kaggle.com/datasets/ikarus777/best-artworks-of-all-time). To be more specific, we subsampled the dataset to fit our computing capacity by only selecting the top artists who have 200 or more paintings. In this way, we get 11 artists(classes) in total, and here is the histogram of the count of paintings for these top artists.
- Additional data source for Cycle GAN Style Transfer (https://www.kaggle.com/datasets/ayaderaghul/gan-getting-started-2)
Here is a table summarizing the train and validation accuracy of the Custom CNN after hyper-parameter tuning and fine-tuned transfer learning model based on ResNet50:
Furthermore, the confusion matrix is shown in the following figure: (The left one is the result of Custom CNN after Hyper-Parameter Tuning and the right one is the result of transfer learning model based on ResNet 50):
Considering the outstanding performance of the transfer learning model based on ResNet 50, here's its prediction result of 4 random images:
To bridge the gap between advanced computational models and end-users, we launched three web interfaces:
- An interface for classification based on the model we trained (custom CNN and transfer learning based on ResNet50),e.g.:
- An interface for the VGG19 and ResNet50-based style transfer.
- An interface for the Monet-inspired CycleGAN style transformations.
Our comprehensive approach not only underscores the adaptability of deep learning in visual tasks but also paves the way for practical real-world applications. Future work might encompass model tuning or adjusting, and exploring other deep learning architectures for enhanced performance.
The code for each task is contained in individual .ipynb files, while the code and samples for each of the three web interfaces are organized in separate folders. Additionally, the project summary can be found in 'ProjectSlides.pdf', and the detailed project report is available in 'ProjectReport.pdf'.