TikTok-Style Video Generator

A powerful tool for automatically generating TikTok-style videos from YouTube content or local audio files. This project processes audio through speech recognition, speaker diarization, and text processing to create engaging short-form videos with animated captions, background visuals, and special effects.

Features

• YouTube Integration: Download and process audio directly from YouTube videos
• Speaker Diarization: Automatically identify and color-code different speakers
• Dynamic Captions: Create animated, pop-in captions synchronized with speech
• Background Processing: Automatically crop and format background videos for TikTok dimensions (9:16)
• Content Moderation: Optional swear word censoring
• Thumbnail Generation: Automatic creation of video thumbnails

Installation

Prerequisites

• UNIX-based system (macOS or Linux, Windows via WSL2)
• ffmpeg (required for audio/video processing)
• ImageMagick (required for image and thumbnail processing)
• Python 3.9–3.13 (tested with 3.13 on Apple Silicon)
• Hugging Face account + access token (needed for diarization models) (note you could skip this if you don't plan on using speaker diarization i.e. coloring the text for different speakers)

macOS: Install system dependencies

brew install ffmpeg imagemagick cmake coreutils pkg-config protobuf

cmake, coreutils, and pkg-config are needed to build certain Python dependencies (sentencepiece in particular). If you hit errors related to nproc or CMake, make sure these are installed.

Linux (Debian/Ubuntu): Install system dependencies

sudo apt update
sudo apt install ffmpeg imagemagick cmake pkg-config libprotobuf-dev protobuf-compiler build-essential

Step-by-Step Setup

1. Clone the repository with submodules

git clone --recursive https://github.com/harry/video-generator.git
cd video-generator

2. Create and configure .env file

cp .env.example .env

Edit the .env file and add your HuggingFace token:

VIDEO_CODEC=h264_videotoolbox  # macOS GPU acceleration; on Linux use libx264
AUDIO_CODEC=aac
HF_TOKEN=your_huggingface_token_here

3. Set up whisper-cpp

For macOS:

chmod +x setup_osx.sh
./setup_osx.sh

Note there is also whisper.cpp available via Homebrew but this code needs adapting first.

For Ubuntu:

chmod +x setup_ubuntu.sh
./setup_ubuntu.sh

4. Create Python virtual environment

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip

Note you need to have the python virtual environment activated for the rest of the setup and when you're generating the videos.

5. Install Python dependencies

We have locked NumPy to >=2.2,<2.3 for OpenCV compatibility and pinned PyTorch/Torchaudio for pyannote.audio.

pip install -r requirements.txt

If you see errors about sentencepiece, preinstall it:

pip install sentencepiece==0.2.0

Usage

Basic Usage

./generate_tiktok.sh [options] <youtube_url | local_directory>

Options

• -h, --help: Show help message
• --debug: Enable debug mode in Python scripts
• --regenerate: Force regeneration of outputs

Input Types

• YouTube URL: e.g., https://www.youtube.com/watch?v=xxxxx
• Local Directory: Path to a directory containing the audio file

Quick Start Example

This example creates a TikTok-style video using Gangnam Style as the background and Carl Sagan's "4th dimension" talk as the audio:

1. Download the background video

mkdir -p background
echo "https://www.youtube.com/watch?v=SW_iujvUAzQ" > background/video-link.txt

The first time generate_tiktok.sh is run, it will download the background video based on the contents of the background/video-link.txt file and process it. This can take a while. Subsequent runs will use the processed background video.

2. Run the generator with Carl Sagan's audio

./generate_tiktok.sh https://youtu.be/UnURElCzGc0

3. Find your generated video

The final video will be available at ./output/<video_id>/final.mp4

Additional Information

For detailed documentation about the project architecture, components, and workflow, please see INTRO.md.

Assets

• The fonts and sample videos included belong to their respective owners.
• You can find more background videos to use here