Packing Input Frame Context in Next-Frame Prediction Models for Video Generation (opens in new tab)

(github.com)

1 pointsvikrantrathore11mo ago2 comments

2 comments

FramePack

    Diffuse thousands of frames at full fps-30 with 13B models using 6GB laptop GPU memory.
    Finetune 13B video model at batch size 64 on a single 8xA100/H100 node for personal/lab experiments.
    Personal RTX 4090 generates at speed 2.5 seconds/frame (unoptimized) or 1.5 seconds/frame (teacache).
    No timestep distillation.
    Video diffusion, but feels like image diffusion.

Paper link: https://lllyasviel.github.io/frame_pack_gitpage/pack.pdf

Website link: https://lllyasviel.github.io/frame_pack_gitpage/

dragonwriter11mo ago

Dupe of: https://news.ycombinator.com/item?id=43736193

j / k navigate · click thread line to collapse

2 comments

vikrantrathoreOP11mo ago

FramePack

    Diffuse thousands of frames at full fps-30 with 13B models using 6GB laptop GPU memory.
    Finetune 13B video model at batch size 64 on a single 8xA100/H100 node for personal/lab experiments.
    Personal RTX 4090 generates at speed 2.5 seconds/frame (unoptimized) or 1.5 seconds/frame (teacache).
    No timestep distillation.
    Video diffusion, but feels like image diffusion.

Paper link: https://lllyasviel.github.io/frame_pack_gitpage/pack.pdf

Website link: https://lllyasviel.github.io/frame_pack_gitpage/

dragonwriter11mo ago

Dupe of: https://news.ycombinator.com/item?id=43736193

j / k navigate · click thread line to collapse