Brainrot or Not

A question from filmmaking

Could a model give me another way to look at a video? I was curious about the possibility of spotting moments that might deserve a different edit, then comparing versions. Brainrot or Not grew out of that question and my interest in trying emerging research for myself.

I built an experimental interface around a pretrained brain-response model, with media inputs, visualizations and my own summary scores. The name is playful; the underlying question is how much useful information I can actually draw from those outputs.

The research underneath

TRIBE v2 is Meta’s multimodal model for predicting fMRI brain responses to video, audio and language. Meta describes training on recordings from more than 700 volunteers. The released inference tools produce predictions for an average subject on a cortical surface, rather than measuring the person watching my interface.

That research supplies the pretrained model. My contribution is the application around it: preparing inputs, invoking the model and exploring how to present its outputs. The animation below is Meta’s explanation of the research, not a recording of my prototype.

Make the model explorable

The Gradio interface exposes three input paths: video, text and image. The preparation code trims video and can turn a still image into a video representation for the model’s event-building path.

The app loads facebook/tribev2, builds the input events and calls the model’s prediction interface. Nilearn utilities then help turn the resulting arrays into brain-surface and atlas visualizations. I kept the steps separate so the media handling, prediction and interpretation can each be examined.

• Python
• Gradio
• TRIBE v2
• Nilearn
• Media preparation

Summarize a large output

A brain map leaves a lot to interpret. I experimented with four manually chosen groups of regions, labelled attention, emotion, memory and reward, to make the output easier to inspect in the interface.

For each group, the scoring function blends 40% mean absolute activation with 60% peak absolute activation, normalizes the result relative to whole-brain statistics and clamps it to a 0–100 display scale. An overall score combines the four groups with weights of 30%, 30%, 20% and 20%, respectively.

Input paths in the interface

3

Custom regional groups

4

Application display scale

0–100

What is built, and what remains open

The reviewed code contains the media-preparation paths, pretrained-model integration, visualization helpers and custom scoring logic. It gives me a concrete way to investigate the idea rather than leave it as a hypothetical service.

I have not validated these custom scores against audience attention, enjoyment or retention. They are exploratory summaries of predicted activity, not measurements taken from viewers watching the clip. I have not collected a viewer study or trained a new neuroscience model for this project.

The next question: does it help an edit?

The useful next step would be to compare model outputs across controlled versions of the same clip, then test whether any differences agree with independent observations from viewers. That would help separate an interesting visualization from something I could use to make an editing decision.

For now, this project connects two things I enjoy: exploring an unfamiliar technical subject and finding ways it might inform creative work. The open question is part of the experiment.