ai

Let’s Make some Art

Learning through experience is one of the best ways to learn. So this is how I went about educating myself about machine learning, artificial intelligence and neural networks. I am an artist and programmer that became interested in how computers could make art. So lets use artificial intelligence to create new artwork.

I decided to use an open source project on GitHub called art-DCGAN by Robbie Barrat. His implementation was used recently in Christie’s first Artificial Intelligence art auction. But I first encountered Robbie Barrat’s work on superRare.co

To start out we need some data to train the neural network. I wanted to use simple drawings to train my first model. I happen to have a library of art that I personally created. They are line drawings made using sound played through an oscilloscope. It turns out that most of the prebuilt code libraries utilize very small image sizes and you need at least 10,000 images to train them effectively.

Get Lots of Images

I happen to have movies of animated oscilloscope line drawings. Most of them come from this artwork called “Baby’s First Mobile”.

To generate a large number of images I turned the videos into individual frames using ffmpeg. Using commands like:

ffmpeg -i babyMobileVoice.mp4 -vf crop=500:500 thumb%04d.jpg -hide_banner

I was able to crop out the center of the video and produce individual frames. After running ffmpeg on all the oscilloscope videos I have and deleting all the black frames I ended up with over 10,000 images.

Make them Small

The next step is to make the images small. Some tools only accept 64 x 64 pixel images. The art-DCGAN uses 128 x 128 pixel images. I borrowed a python script from this blog at coria.com.   To double the number of images I read that someone rotated them by 90 degrees for additional training examples. Using these directions I added a rotate feature and made a copy of each image.

import sys
import os
import numpy as np
from os import walk
import cv2
import imutils

# width to resize
width = int(sys.argv[1])
# height to resize
height = int(sys.argv[2])
# location of the input dataset
input_dir = sys.argv[3]
# location of the output dataset
out_dir = sys.argv[4]

if len(sys.argv) != 5:
print("Please specify width, height, input directory and output directory.")
sys.exit(0)

print("Working...")

# get all the pictures in directory
images = []
ext = (".jpeg", ".jpg", ".png")

for (dirpath, dirnames, filenames) in walk(input_dir):
for filename in filenames:
if filename.endswith(ext):
images.append(os.path.join(dirpath, filename))

for image in images:
img = cv2.imread(image, cv2.IMREAD_UNCHANGED)
print(image) 
h, w = img.shape[:2]
pad_bottom, pad_right = 0, 0
if h < w:
ratio = w / h
else:
ratio = h / w

if h > height or w > width:
# shrinking image algorithm
interp = cv2.INTER_AREA
else:
# stretching image algorithm
interp = cv2.INTER_CUBIC

w = width
h = round(w / ratio)
if h > height:
h = int(height)
w = round(h * ratio)
pad_bottom = abs(height - h)
pad_right = abs(width - w)
print(w)
print(h) 
scaled_img = cv2.resize(img, (w, int(h)), interpolation=interp)
padded_img = cv2.copyMakeBorder(
scaled_img,0,pad_bottom,0,pad_right,borderType=cv2.BORDER_CONSTANT,value=[0,0,0])

cv2.imwrite(os.path.join(out_dir, os.path.basename(image)), padded_img)
rotate_img = imutils.rotate(padded_img, 90)
cv2.imwrite(os.path.join(out_dir, '90_'+os.path.basename(image)), rotate_img) 

print("Completed!")

Running this command

python resize_images.py 128 128 /origin /destination

will produce images of the proper size for the training to begin.

Setup an AWS GPU Instance

The GitHub art-DCGAN project has install directions but I will add to them here. In AWS I selected:

Install all the Software

DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=50 ngf=150 th main.lua

DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=12 ngf=150 th main.lua

Now the Artificial Intelligence model will make some new art

batchSize=36 net=./checkpoints/scope_90_net_G.t7 th generate.lua

Artificial Intelligence Generated Oscilloscope Art

AI Generated Oscilloscope Art

Now it becomes a matter of picking out the best work as an AI art curator.

With all of this work it becomes obvious that this is a very slow way to make art. It is still very much a human endeavor. But allowing a computer to generate lots of ideas for an artist to explore is helpful. Many artists have used randomized systems to generate art ideas. Artists such David Bowie, Brian Eno, Nam June Paik and John Cage have used mechanical and computer generators to automate some of their work and kick start the creative process.

In addition by making this model we have learned how to train other models that can be used for image recognition. You can also try out the other features of art-DCGAN that will download art from a public site and then generate new images based on that genre.