The system watches the movie frame by frame. For every face that appears, it detects the identity and reads the emotion. For every line spoken, it first separates vocals from background music (Demucs), reduces noise (DeepFilterNet), then transcribes every word with millisecond-precision timestamps (WhisperX). It identifies who is speaking (pyannote diarization) and measures their vocal emotion — arousal, valence, dominance (Wav2Vec2 SER). The transcribed text is then run through DistilRoBERTa emotion classification to get a text-level emotion label and confidence score — a third independent signal alongside face and voice. Three channels, each telling a potentially different story about the same moment.

A smile means nothing without context. Is she genuinely happy, or masking pain? The only way to know is to remember what happened before. Our system maintains two layers of memory for each character: short-term memory tracks the current scene (sliding window of recent events), while long-term memory spans the entire film (stored in ChromaDB vector database). When something surprising happens — measured by an LSTM model — it gets promoted from short-term to long-term storage.

This is where perception and memory merge into understanding. When the face says happy, the voice says distressed, and the text says sad — that's not noise, that's a signal. The system classifies this as an emotional conflict, indicating possible suppression. It also builds long-term character models: Big-5 personality traits (OCEAN) drive predicted behavior baselines; Dunbar social circles map relationship intimacy; and a 6-dimension scoring engine evaluates acting performance with transparent, reproducible formulas.

The final layer transforms analysis into action. The system generates a 6-dimension prompt for each key scene: expression, body language, atmosphere, camera, composition, and dialogue action. Pipeline data is the primary driver — emotion labels, arousal/valence, and Dunbar circles directly determine every dimension. Screenplay stage directions serve only as supplementary reference for physical actions that sensors cannot capture (e.g., "walks toward the door"). Three of six dimensions are 100% pipeline-driven with no screenplay input at all. Without the screenplay, the system still works — our ablation experiment shows pipeline-only prompts produce coherent videos, just with less scene detail. The video output is proof that our pipeline analysis is specific enough to be actionable.