video-killed-the-radio-star

# Video Killed The Radio Star [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dmarx/video-killed-the-radio-star/blob/main/Video_Killed_The_Radio_Star_Defusion.ipynb)



## Requirements

* ffmpeg - https://ffmpeg.org/
* pytorch - https://pytorch.org/get-started/locally/
* vktrs - (this repo) - `pip install vktrs[api]`
* stability_sdk api token - https://beta.dreamstudio.ai/ > circular icon in top right > membership > API Key
* whisper - `pip install git+https://github.com/openai/whisper`

## FAQ

**What is this?**

TLDR: Automated music video maker, given an mp3 or a youtube URL

**How does this animation technique work?**

For each text prompt you provide, the notebook will...

1. Generate an image based on that text prompt (using stable diffusion)
2. Use the generated image as the `init_image` to recombine with the text prompt to generate variations similar to the first image. This produces a sequence of extremely s