Autoregressive Multi-Shot Video Generation

CineMem: Attention-Aligned Fine-grained Memory Retrieval for Multi-Shot Long Video Generation

Xiaoxue Wu1,2*,

Xinyu Gao3,

Fang Yu1,4,

Yu Qiao1†,

Xinyuan Chen1†,
1Shanghai Artificial Intelligence Laboratory  2Fudan University  3Shanghai Jiao Tong University  4East China Normal University 
*Work done during internship at Shanghai AI Laboratory †Corresponding author
Online-clustered KV cache Query-adaptive retrieval Long-range inter-shot consistency
Paper

Paper Summary

Abstract

Autoregressive video generation relies on memory retrieval to model long-range dependencies. This capability becomes particularly critical for multi-shot generation, where models must reconcile substantial visual changes across shot transitions with persistent semantic continuity. Existing methods tend to perform memory retrieval at the frame level and share the same historical context across all query tokens, overlooking the heterogeneous information needs and leading to redundant retrieval and inefficient use of historical memory. To bridge this gap, we introduce CineMem, an attention-aligned fine-grained memory retrieval framework that combines online memory organization with query-adaptive retrieval for autoregressive multi-shot video generation. Specifically, guided by intrinsic memory-access patterns revealed through attention analysis, we propose an online-clustered KV cache as a memory organization strategy over all historical tokens, enabling semantic-aware memory grouping and more precise retrieval. In addition, we introduce a query-level retrieval mechanism that provides broader access to historical information under a fixed retrieval budget, thereby improving long-term memory utilization. Together, these designs enable the multi-shot video generator to preserve long-range consistency across shot transitions, particularly in revisitation scenarios, while approximating full-memory access under a constrained memory budget. To facilitate evaluation, we further introduce a suite of task-specific, VLM-assisted metrics for faithful inter-shot consistency assessment. Extensive experiments demonstrate the effectiveness of our framework.

Qualitative Results

Autoregressive Multi-Shot Generation

These examples illustrate how fine-grained memory retrieval helps preserve subject identity, scene semantics, and visual continuity across shot transitions.

Fine-Grained Memory

CineMem

Autoregressive video generation depends on memory retrieval to model long-range dependencies. CineMem addresses the heterogeneous information needs of query tokens with online memory organization and query-adaptive retrieval, enabling more precise and less redundant access to historical context under a fixed retrieval budget.

1
Online-clustered KV cache
Organize all historical tokens into fine-grained memory groups for more precise retrieval.
2
Query-adaptive retrieval
Give each query token broader access to historical information within a fixed retrieval budget.
3
Inter-shot consistency
Preserve long-range semantic continuity, especially when the generation revisits earlier content.
Method overview placeholder

CineMem overview: online memory organization and query-adaptive retrieval

Baselines and Analysis

Comparison

Each case contains eight videos generated from the same prompt. The case prompt and shot count are shown above its comparison grid.

BibTeX

@misc{wu2026cinememattentionalignedfinegrainedmemory,
      title={CineMem: Attention-Aligned Fine-grained Memory Retrieval for Multi-Shot Long Video Generation}, 
      author={Xiaoxue Wu and Xinyu Gao and Fang Yu and Yu Qiao and Xinyuan Chen},
      year={2026},
      eprint={2610.17926},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.17926}, 
  }
-->