This article is part of a series about upscaling icon themes with AI. While this page is part 4, part 3 is available here and part 5 is available here.
Pixel Pal is a project I have started to bring new life to old icon themes. Many old icon themes are in raster format and are not suited to HiDPI screens. Currently, the only way to handle old icon themes is to turn one pixel into four or nine, but the result looks pixelated. The goal of this project is to use deep learning to upsample small icons into icons for HiDPI screens.
One of the big challenges in deep learning is getting enough data. Neural networks can easily have millions of parameters, and fine-tuning them requires a large quantity of data. Getting enough data is a challenge, but what makes it worse is that data alone is not enough: you need labelled data and, if possible, high-quality labelled data.
When you don’t have enough data, you have a problem. There are many ways to tackle this issue:
- Reducing the number of parameters to learn. One way is to reuse the weights (parameters) from a model already trained on another task, and then retrain only parts of it. This is a form of transfer learning. For instance, VGG-16 is trained for general-purpose image classification. You can take this model and retrain only parts of it, assuming it generalise well.
- Training parts of the model on the input data alone using autoencoders. This is fairly technical. It is a model which compresses the input and then decompresses it with as little loss of quality as possible. Such a model is easier to train, because you usually have lots of input data (but not necessarily output data). Then you take the trained autoencoder and split it in two: the compression network and the decompression network. You can then work with the compressed features coming out of the compression network in place of the original input data. The benefit is that the compressed features are smaller. As a result, the network you need to train to map the compressed features to the output data is also smaller, and therefore simpler to train.
- Using data augmentation. This consists of building new data from the existing data, and it is especially good at producing neural networks that generalise well. Through data augmentation, you can tell a neural network what it should not rely on. For instance, one way to augment image data is to flip the image horizontally and vertically; this way, you can easily quadruple the amount of data available and tell the network it should not rely on the image being a certain way, like having shadows at the bottom.
For this project, I will rely on data augmentation. My training data set is around 9,000 images, which is fairly good for networks with tens of thousands of parameters. But through data augmentation, my neural network can do even better: by simply flipping horizontally and vertically, I can easily get around 36,000 images, which allows me to investigate deeper neural networks.
| Modification | No horizontal flip | Horizontal flip |
| No vertical flip |  |  |
| Vertical flip |  |  |
