# Video outpainting > Expand a video’s canvas by generating the missing regions, with a focus on consistency from frame to frame. Personal project · Experimental tooling Category: Tools & research [Open in the interactive portfolio]() [Français]() ## At a glance | Metric | Value | | --- | --- | | My role | Prototype & integration | | Goal | Adapt a video’s format | | Challenge | Consistency over time | | Stage | Experimental prototype | ## Change the format while keeping the scene The end goal was to adapt footage to another frame by generating what is missing around the image. That can mean widening a 4:3 video to 16:9 or preparing a portrait composition from a landscape shot. My prototype initially focused on the second case: placing landscape footage inside a portrait canvas and filling the areas above and below it. The 4:3-to-16:9 example below explains the general idea; it is not a measured result from my code. ### Images and media [At the same height, the source occupies 75% of the 16:9 frame and each added side band 12.5%. These are geometric proportions, not model-quality scores.]() Diagram showing the retained 4:3 source in the middle of a 16:9 target with newly generated side bands. Credit: Explanatory diagram created for this portfolio. No generated output is implied. ## What video outpainting looks like Seen-to-Scene’s examples make the idea visible on real sequences: observed content remains recognizable while the visible field is extended. The examples below come from InSeok Jeon and his coauthors’ research project. They illustrate the kind of outcome I wanted to work towards, rather than outputs from my prototype. I did not implement or train Seen-to-Scene; my project explores its own orchestration of diffusion and motion-guided propagation. ### Images and media [External illustration: source sequences and expanded results published by the Seen-to-Scene authors. These are not my results.]() Seen-to-Scene montage of dog and motocross sequences, showing source views and expanded results. Credit: [© 2026 InSeok Jeon and coauthors · Seen-to-Scene · MIT-licensed repository.]() [External demonstration from the authors’ GIF, converted to a click-to-play preview. Its 25 frames are presented at 10 frames per second.]() Grid of eight animated video-outpainting examples published by Seen-to-Scene. Credit: [InSeok Jeon and coauthors · Seen-to-Scene, 2026. Authors’ source and method.]() ## The hard part: keeping the surroundings consistent A single expanded frame can look convincing. In a video, textures, objects and lighting also need to stay coherent as the camera or subjects move. Generating every frame independently can produce surroundings that change or flicker. The pipeline uses estimated motion to warp the preceding generated frame and initialize the next fill. It also computes a propagated latent-noise state, but this version does not yet pass that state into the inpainting call. That part of the intended continuity mechanism remains unfinished. ## Build a pipeline I can inspect The code separates frame extraction, canvas and mask preparation, motion estimation, propagation, filling and clip reconstruction. It also provides a deflicker stage and audio reassembly. Generation uses a Diffusers inpainting integration. A deterministic fill mode lets me test the pipeline mechanics without running the model. The stages are configurable so I can isolate a problem or replace an implementation. Python · Diffusers · Optical flow · Latent propagation · SSIM / optional LPIPS A clip’s path through the prototype 1. **Prepare** — Extract frames, build the canvas and mask the areas to fill. 2. **Propagate & generate** — Estimate motion, carry preceding state and fill the missing areas. 3. **Inspect & assemble** — Collect diagnostics, apply post-processing and rebuild the video. Summary of the code reviewed for this entry. This describes my prototype, not Seen-to-Scene’s architecture. ## What I tested, and what remains One saved short run contains three adjacent-frame comparisons: mean SSIM 0.9239 and mean flow-warped temporal error 3.5450. These values help inspect that small run. They do not mean “92% visual quality” and are not a controlled comparison against another method. The end goal remains a format-adaptation tool I could evaluate across varied scenes, with the source frame and generated regions clearly identified. I still need representative outputs across a wider range of scenes to judge the visual result. | Value | Metric | | --- | --- | | 3 | Comparisons in the short run | | 0.9239 | Mean SSIM · local short run | | 3.5450 | Mean temporal error · code-specific scale | > Diagnostics come from an existing result file reread on 9 November 2026. No training or new generation was run. The exact settings for this run and a comparison baseline are not established. ## Links - [Seen-to-Scene · visual examples & source]() - [Seen-to-Scene · MIT license]()