Listing Updated - March 2023 🤖

Riffusion

Riffusion is a music generator that creates music from text prompts, allowing users to experiment with different styles, instruments, genres, and sounds to explore the latent space of sound. The platform uses a base for img2img diffusion to keep riffs on beat and impact melodic patterns. Users can adjust the settings to explore and create different musical outcomes, with higher denoising resulting in more creative output, but also potentially causing the music to be off-beat.

Features:

  • Generates music from text prompts
  • Supports a wide range of styles, instruments, modifiers, genres, and sounds
  • Can generate infinite variations of a prompt by varying the seed
  • Image-to-image conditioning allows for modifying sounds while preserving the structure of an original clip
  • Smoothly interpolates between prompts and seeds in the latent space of the model
  • Interactive web app with real-time generation and 3D visualization of spectrogram timeline

Technical Details:

  • Built on Stable Diffusion, the open-source AI model that generates images from text
  • Fine-tuned on images of spectrograms paired with text to generate audio clips
  • Short-time Fourier transform (STFT) used to compute the spectrogram from audio
  • Griffin-Lim algorithm used to approximate the phase when reconstructing the audio clip
  • Torchaudio used for efficient audio processing on the GPU

Cost and Availability:



‍

‍

Use Cases

Interactive Media

Riffusion can be used in interactive media such as websites and apps to provide users with real-time generated music that responds to their inputs. The interactive web app with real-time generation and 3D visualization of spectrogram timeline can be embedded in interactive media to provide a unique audio-visual experience.

Sound Design

Riffusion can be used in sound design to create custom sounds and effects for films, video games, and other media projects. The ability to modify sounds while preserving their structure allows for the creation of new sounds that are still recognizable as variations of existing ones.

Music Production

Riffusion can be used in the music industry for producing original music tracks. It allows users to experiment with various styles, instruments, genres, and sounds to create unique compositions.

Related Tools

Listing Updated - March 2023 🤖

boomy

Free Tier? Yes Without A Credit Card

Listing Updated - March 2023 🤖

Mubert

Free Tier? Yes Without A Credit Card

Find Out How How Best To Utilise AI Tools

Our newsletter comes with exclusive discounts, trials and practical insights from within the industry