I've seen a bunch of people share their AI art generators, so I figured I'd get in on the action and share mine. This is based on the notebooks written by AI artists like advadnoun and Katherine Crowson. The difference between those and this is:
- Friendlier UI/UX: I used colab forms and IPython widgets to make it easy to use if you don't know how to program. For example, instead of having people fiddle with sampling vs pooling methods, you can just select "default" or "cumin" flavors.
- Simplified inputs: I stripped out a lot of the power from "artist-oriented" notebooks to make it less scary to first timers. You can still edit the code directly to get the same power, but all that's hidden away.
- "Download all images" and "create a video" buttons for funsies.
- Totally free. This is running off Google's free Colab offering. You just need a Google account.
- No wait times. Colab gives a separate GPU to each user, so there's no queueing or anything.
- A bit more power: if Colab assigns you a large instance (most people are getting large instances, unless you use it for a very long time), you can make images up to 700x700.
- Codes all available if you want to clone it, make your own version, or download it to run locally.
I shared this on Twitter last week and lots of nonprogrammers are able to use it, which tells me that it works pretty well at its intended goals. So far this is my favorite thing anybody's made with it (video, sound): https://twitter.com/PuPunPa/status/1423428815846481924
Also happy to answer any questions people have about AI art in general. I only fell down the rabbit hole last month, and it's a deep rabbit hole, so I can't promise much, but I really like talking about this stuff and sharing what I've learned. Enjoy!
> A bit more power: if Colab assigns you a large instance (most people are getting large instances, unless you use it for a very long time), you can make images up to 700x700.
Technically you just need a GPU with 16GB VRAM to create larger images, which is every GPU Colab can give except the K80 (which has 12GB VRAM, but still can do 600x600)
Your Notebook code appears to do to a VRAM check via nvidia-smi, not an instance size check.
Does anybody know of the best practice for creating large images (ex 3840x2160) on a single GPU (ex a 2080/3080)? I played with stitching some years ago when all of this stuff was still in its infancy, but wasn't super thrilled with the results as I ended up with seams, now I'm wondering if a common practice has emerged as the techniques have evolved.
What you could try doing is use a notebook that lets you start with an initial image [1], chunk that image, and generate a an image for each chunk. It wouldn't be unified, but it could come out interesting.
To make bigger images, I'm currently making composites out of multiple smaller prompts, like gradients or chessboards.
I know HN has special rules for formatting certain URLs. Like Twitter links get the username added on and apparently research.google.com gets a subdomain (Or maybe all of Google?).
Turns out this link is actually from colab, a repository of user-generated content hosted at colab.research.google.com.
@dang: if it isn't too unwieldy, maybe colab links could have the 'colab' part added on too? The way it is now led me to think the link led to an official Google Research publication/project.
See also my notebook (which was released after the OP's notebook and I was not aware of it until after I completed mine) from the same source which focuses on more streamlining and more deterministic output by using icon-based input/target images: https://colab.research.google.com/drive/1wkF67ThUz37T2_oPIuS...
I think we're in a "lightbulb" moment: the tech has gotten good and convenient enough that a lot of people are having the same ideas. I mean, heck, I don't know anything about AI or art and I was still able to make my notebook. There's also been a bunch of paid solutions submitted to HN, too, as people try to find ways to build products off this.
The great thing is that the space is big and crazy enough that even if a bunch of people independently make something, everything they make will be different in interesting ways. I'm just focusing on making it as simple as possible, to the point where I don't even have an option to have different width and height. I give it maybe a month before someone makes a Colab that blows both of ours out of the water.
(I should really start aggregating the notebooks, so people who enjoyed mine and want to try all the other cool ones can see them all in one place)
I found it really interesting that people discovered that you can get improve quality by including strings like "Blender HD", "Unity Engine", "HD Wallpaper".
I’ve been utterly obsessed with the VQGAN notebooks since they came out and had a ton of fun.
It’s amazing how much difference choosing the right prompts can make - trending on artstation, by James Gurney (even James Gurney himself got in on it), unreal engine, volumetric lighting etc.
No hyperbole in that this is one of the biggest things to happen in digital art, probably since Photoshop.
I recently wrote up some of my experiments, observations, and findings with VQGAN+CLIP (the models folks have been using to do this AI Art recently) here: https://blog.roboflow.com/ai-generated-art/
> I found myself wanting to more actively steer. Changing the prompt in the middle of a generation to "take a right turn" would be interesting to experiment with.
I've tried this! You get weird stuff. I think of it like glitch art. One idea I want to try is directly fiddling with the tensors mid-run, like rotating it along one axis or reorder columns or something. I think you could get really interesting stuff that way
I was playing around with this with some friends and I have to say that anything involving animals of any kind in the generation leads to really weird results. I haven't tried other AI art programs, so I don't know of this is sort of expected, but definitely interesting.
Same thing with trees. It definitely seems to struggle more with individual objects you'd expect to see in lots of photos. It's really good at locations, though.
(Some of that might also be what I hardcoded. Most notebooks give a lot more flexibility in the art you produced, I simplified things dramatically for accessibility purposes)
Am I a total idiot as I am completely unable to get any output from this?
I mean, I spent about 5 minutes reading the various inputs and trying to modify them, yet I am baffled as what to do to actually see, or even know what's going on, what the system is producing.
The https://heibox.uni-heidelberg.de URL required for the checkpoint file seems to be down or at least very flaky. I wonder how many Colab boxes are currently trying to download the 900 megabyte weight file...
Had fun with this yesterday, sadly today I'm getting "RuntimeError: unexpected EOF, expected 733 more bytes. The file might be corrupted." Hope it can be fixed, it was fun exploring what it knew about art and the world!
- Friendlier UI/UX: I used colab forms and IPython widgets to make it easy to use if you don't know how to program. For example, instead of having people fiddle with sampling vs pooling methods, you can just select "default" or "cumin" flavors.
- Simplified inputs: I stripped out a lot of the power from "artist-oriented" notebooks to make it less scary to first timers. You can still edit the code directly to get the same power, but all that's hidden away.
- "Download all images" and "create a video" buttons for funsies.
People have also been posting commercial AI offerings, like https://news.ycombinator.com/item?id=28070149. The differences with this are:
- Totally free. This is running off Google's free Colab offering. You just need a Google account.
- No wait times. Colab gives a separate GPU to each user, so there's no queueing or anything.
- A bit more power: if Colab assigns you a large instance (most people are getting large instances, unless you use it for a very long time), you can make images up to 700x700.
- Codes all available if you want to clone it, make your own version, or download it to run locally.
I shared this on Twitter last week and lots of nonprogrammers are able to use it, which tells me that it works pretty well at its intended goals. So far this is my favorite thing anybody's made with it (video, sound): https://twitter.com/PuPunPa/status/1423428815846481924
Also happy to answer any questions people have about AI art in general. I only fell down the rabbit hole last month, and it's a deep rabbit hole, so I can't promise much, but I really like talking about this stuff and sharing what I've learned. Enjoy!