Let’s Make some Art
Learning through experience is one of the best ways to learn. So this is how I went about educating myself about machine learning, artificial intelligence and neural networks. I am an artist and programmer that became interested in how computers could make art. So lets use artificial intelligence to create new artwork.
I decided to use an open source project on GitHub called art-DCGAN by Robbie Barrat. His implementation was used recently in Christie’s first Artificial Intelligence art auction. But I first encountered Robbie Barrat’s work on superRare.co
To start out we need some data to train the neural network. I wanted to use simple drawings to train my first model. I happen to have a library of art that I personally created. They are line drawings made using sound played through an oscilloscope. It turns out that most of the prebuilt code libraries utilize very small image sizes and you need at least 10,000 images to train them effectively.
Get Lots of Images
I happen to have movies of animated oscilloscope line drawings. Most of them come from this artwork called “Baby’s First Mobile”.
To generate a large number of images I turned the videos into individual frames using ffmpeg. Using commands like:
ffmpeg -i babyMobileVoice.mp4 -vf crop=500:500 thumb%04d.jpg -hide_banner
I was able to crop out the center of the video and produce individual frames. After running ffmpeg on all the oscilloscope videos I have and deleting all the black frames I ended up with over 10,000 images.
Make them Small
The next step is to make the images small. Some tools only accept 64 x 64 pixel images. The art-DCGAN uses 128 x 128 pixel images. I borrowed a python script from this blog at coria.com. To double the number of images I read that someone rotated them by 90 degrees for additional training examples. Using these directions I added a rotate feature and made a copy of each image.
import sys import os import numpy as np from os import walk import cv2 import imutils # width to resize width = int(sys.argv) # height to resize height = int(sys.argv) # location of the input dataset input_dir = sys.argv # location of the output dataset out_dir = sys.argv if len(sys.argv) != 5: print("Please specify width, height, input directory and output directory.") sys.exit(0) print("Working...") # get all the pictures in directory images =  ext = (".jpeg", ".jpg", ".png") for (dirpath, dirnames, filenames) in walk(input_dir): for filename in filenames: if filename.endswith(ext): images.append(os.path.join(dirpath, filename)) for image in images: img = cv2.imread(image, cv2.IMREAD_UNCHANGED) print(image) h, w = img.shape[:2] pad_bottom, pad_right = 0, 0 if h < w: ratio = w / h else: ratio = h / w if h > height or w > width: # shrinking image algorithm interp = cv2.INTER_AREA else: # stretching image algorithm interp = cv2.INTER_CUBIC w = width h = round(w / ratio) if h > height: h = int(height) w = round(h * ratio) pad_bottom = abs(height - h) pad_right = abs(width - w) print(w) print(h) scaled_img = cv2.resize(img, (w, int(h)), interpolation=interp) padded_img = cv2.copyMakeBorder( scaled_img,0,pad_bottom,0,pad_right,borderType=cv2.BORDER_CONSTANT,value=[0,0,0]) cv2.imwrite(os.path.join(out_dir, os.path.basename(image)), padded_img) rotate_img = imutils.rotate(padded_img, 90) cv2.imwrite(os.path.join(out_dir, '90_'+os.path.basename(image)), rotate_img) print("Completed!")
Running this command
python resize_images.py 128 128 /origin /destination
will produce images of the proper size for the training to begin.
Setup an AWS GPU Instance
The GitHub art-DCGAN project has install directions but I will add to them here. In AWS I selected:
Install all the Software
DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=50 ngf=150 th main.lua
DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=12 ngf=150 th main.lua
Now the Artificial Intelligence model will make some new art
batchSize=36 net=./checkpoints/scope_90_net_G.t7 th generate.lua
Now it becomes a matter of picking out the best work as an AI art curator.
With all of this work it becomes obvious that this is a very slow way to make art. It is still very much a human endeavor. But allowing a computer to generate lots of ideas for an artist to explore is helpful. Many artists have used randomized systems to generate art ideas. Artists such David Bowie, Brian Eno, Nam June Paik and John Cage have used mechanical and computer generators to automate some of their work and kick start the creative process.
In addition by making this model we have learned how to train other models that can be used for image recognition. You can also try out the other features of art-DCGAN that will download art from a public site and then generate new images based on that genre.