Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Blotter – An interactive, never ending music video (twitch.tv)
166 points by ck019 on May 22, 2023 | hide | past | favorite | 40 comments
One day I was listening to a playlist and wished there could be some cool visuals to go along with it.

Blotter is a proof of concept I hacked together that does a bit of audio recognition combined with a few generative AI models (both text and img) to create visuals that are relevant to the song.

The video stream is generated in real time at 24fps - you can try it yourself by requesting visuals in the Twitch chat using the "!v" command!

Right now it's mostly a fun hack project, but I am tinkering with new model architectures for higher fidelity video as well as an interactive tool so people can make videos with their own audio files.

I'd love to hear any feedback or suggestions, thanks!



A neverending stream of dynamic content based on audience interaction. It's way more entertaining than I anticipated. This should also be fairly easy to monetize - either by having people donate for interaction or by including ads.

This feels like a solid glimpse of the truly new things that we'll likely see take shape soon.


Definitely wild to see how far the tech has come in even the past few months! Thanks for checking out the stream!


Would be neat to see the distribution of time that people stare at this... I imagine some will stare for hours


I've definitely fallen into that trap before lol


I can't do generated images, especially like in this video. Makes my head hurt or makes me nauseous, either way I can't imagine who would really watch nothing but unfiltered image gen like this. I think you'd go insane trying to make sense of it.


Totally get that!

Thanks for taking the time to check it out either way!


i get that for the first few seconds until my brain starts to filter the jitteriness

is there a way to apply a little stabilization to consecutive outputs, something like https://www.youtube.com/watch?v=X1WG5jc5SU0 ?


This is so fun, although it is super grainy. Is it because you cannot run the generation at higher res in time?


Yep that's right! This project was definitely an exercise in balancing visual fidelity and inference speed. I have a couple of ideas to increase realism though that I'm excited to dig into and hopefully have live soon.

Thanks for checking out the stream!


It's interesting to see what people come up with in the chat, other than the typical Shrek meme prompts and the like. How do people even come to think of stuff like "demon core party prank"? Wild.


It was definitely as entertaining watching chat as it was watching the visuals!

That being said, one of my biggest takeaways from this whole experience was I could've just trained a Shrek generator and it would've covered like 90% of use cases


I am currently learning how to mix house music. Would totally use something like this to keep my audience visually engaged.


That's amazing, how can I subscribe and follow your next improvements on this?


A really interesting project. I would try to leave this playing tonight as a screensaver.


Thanks for checking out the stream!


You made the ultimate VJ party tool.


AI tool killed the VJ-stars' gig (with a nod to "The Buggles")


Woo! Thanks for checking it out


nice idea, weird visuals, I'd prompt it to stay in the abstract zone maybe... twitch now has What The Robot Saw, WatchMeForever, AI Jesus, and my channel m88t where I run an AI Nursery...


The music fits this project well. Is it always a lofi channel?


Thanks for checking it out! Yeah right now it's all lofi - was thinking about making a few different streams for different genres though


Really nicely done, the visuals really react well to the music. I am myself not a big fan of lofi but can imagine having this on as a nice background with some ambient music.


Cool, but I will stick to this stream, thanks. Loving it. Stream name is inspired by little squares of cardboard?


Haha indeed it is


Fun! Mind sharing which ML models you used?


Yeah! It's actually a fair bit of custom code tying together parts of multiple models.

Hoping to clean it up a bit and share an e2e repo soon


I'd be happy to even see the un-clean repo.


Cool that it knows other languages.


Are you going to release the code?


Yeah! It'd be cool to see what others build on top of this.

Gonna add cleaning things up a bit and writing some documentation to my to-do list


Appreciate that, I would love to tinker and play with it and see what comes out.

Always loved things that go in time with music


Very keen on this. Expressed interest on your web form


It's pretty fun


Love to hear it! Thanks for checking out the stream


what kind of machine are you running this on?


g4dn.2xlarge

Thanks for checking out the stream!


When I opened it someone requested visuals of Biden and Putin kissing.

Slippery slope I say this.


> Slippery slope I say this.

Or humans need to chill out, realize we're a bunch of primates on a space rock, and stop being so offended and afraid.

So what if we use politicians, world leaders, and celebs? This is no different than photoshop.

You know what's funny?

https://www.youtube.com/watch?v=IkaAZE_UGMo

https://www.youtube.com/shorts/ivhUH68KWis

Or something along those lines:

https://www.youtube.com/watch?v=03vmbKGW1iI

This is even funnier to me because I have no idea what any of this means. Kids love this stuff, though, and they're wildly creative.

We're just getting started with the creative Cambrian explosion. We have yet to hit the inflection point.


Do you think twich should have moderation?


I think Twitch should be a protocol.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: