ARTificial INTELLIGENCE

Changes

Home | Changes | About

I saw Nick Gregory's diffusion.gallery site in the fall of 2022. Every five minutes, it uses the same simple prompt to an OpenAI model to generate a description of a painting. It then uses Stable Diffusion to generate an image from that description. It is a simple idea, but the results are fantastic: constantly refreshing, never-before-seen works of art. I have eight displays mounted to the walls in my office for Grafana, but have been mostly unused lately. I added diffusion.gallery to all eight displays attuned to a different points in diffusion.gallery's history of paintings. I love it!

Some time later, I came across the wonderful Stable Diffusion 1.5 - Cheat Sheet. While diffusion.gallery has some variety, the huge list of artists in the Cheat Sheet showed me I could have so much more! New, better models were available and starting with a dynamic initial prompt based off artists details from the Cheat Sheet meant I should be able to have more variety with fewer unsuccessful mutated pictures.

Generator

Nick's approach was to run his models on a local machine. I opted to call APIs, which costs more, but makes it easier to call more models and update to newer versions faster. The generator only runs every 20 minutes to keep costs reasonable. Here's how it works:
  1. Every 20 minutes, an Amazon Eventbridge schedule triggers the AWS Lambda generator function.
  2. The generator chooses a basic genre, like "landscape" or "architecture".
  3. Using the Cheat Sheet artist data (now heavily modified), the generator chooses an artist tagged with that genre then randomly chooses methods, subjects, styles, etc. to assemble a prompt for Stable Diffusion.
  4. For most subjects, it pulls from a queue of subjects generated by GPT specific to that artist's works. When the queue runs out, it asks GPT for 20 more subjects.
  5. The generator sends the prompt to the StabilityAI API to generate an image using the Stable Diffusion 3 model. Previous versions used the SDXL 1.0, SDXL beta model, SD 1.5 model, Deliberate V2 checkpoint, and Dreamshaper checkpoint.
  6. The image is stored in Amazon S3 and all the prompt info and image metadata is stored in Amazon DynamoDB.

Website

Everything is hosted at AWS. The website uses CloudFront as CDN and webserver to serve pages and assets from S3 and route API calls to API Gateway. API Gateway calls a Lambda function to look up info on the next image from DynamoDB so the website will know what image to pull from S3.
Having written one of the first enterprise level CMSes, what CMS/website builder did I use? I wrote the HTML and CSS by hand in a text editor. Probably not the most efficient approach, but there are only a handful of pages. And I have control over what the website is doing, so I can keep it from doing any of the stupid tracking that is built into almost every website. No cookies. No local browser storage. There are logs, but those are for troubleshooting and clear after a few days.
The style of the website comes from Tufte CSS.