Memory-Efficient Long-Video Understanding (Capstone)

Lee DongWon · Sep. 2026 – Present

Overview

With streaming long video, the KV cache grows with the number of frames: Qwen2-VL-7B at 768 frames holds about 50K tokens of full KV. InfiniPot-V compresses it to a 6K budget. I store the kept KV cache in INT4 instead of FP16. At the same 6K budget, INT4 reaches Video-MME 63.5 vs 62.8 and MLVU 66.1 vs 65.8 (FP16), using about 1/3.1 the KV bytes (164 B vs 512 B per token per head).

Approach

InfiniPot-V (TaR + VaN + Pool) compresses the KV cache to a 6K budget.
The kept KV is stored as INT4 KIVI: keys quantized per channel, values per token, group size 32, and the most recent 128 tokens kept in FP16.
Compared against FP16 at 6K, at equal KV bytes.
Model: Qwen2-VL-7B. Benchmarks: Video-MME and MLVU.

My Role

Added INT4 KIVI KV storage on top of InfiniPot-V.
Ran ablations, using the best setting (including Pool) as the baseline.
Built the Video-MME / MLVU evaluation pipeline on a Slurm GPU cluster.

Tech: PyTorch, KV-cache quantization.

Results

Table 1. Main results (6K budget, same KV bytes)
Method	KV	Budget	Video-MME S/M/L/Avg	MLVU Hol/Single/Multi/Avg
Full KV (InfiniPot-V paper)†	FP16	50K	74.7 / 62.1 / 55.0 / 63.9	76.3 / 73.9 / 43.3 / 65.9
Full KV (ours)	FP16	50K	– / – / – / – ([TBD])	76.1 / 74.8 / 43.1 / 66.1
InfiniPot-V (paper)†	FP16	6K	74.1 / 60.8 / 53.4 / 62.8	77.2 / 72.3 / 44.8 / 65.8
InfiniPot-V reproduced (TaR+VaN+Pool)	FP16	6K	74.6 / 61.4 / 52.4 / 62.8	78.9 / 71.5 / 44.0 / 65.8
Ours (TaR+VaN+Pool + INT4 KIVI)	INT4	6K	74.2 / 62.0 / 54.2 / 63.5	77.6 / 72.7 / 44.8 / 66.1

† From the InfiniPot-V paper. Full KV rows are reference rows (50K budget). “Full KV (ours)” uses Qwen2-VL-7B with FP16 KV at 50K, over 2,174 questions. Bold marks the best value per column among the 6K methods (Full KV rows excluded); ties are all bold.

Video-MME FP1662.8

Video-MME INT463.5

MLVU FP1665.8

MLVU INT466.1

Average accuracy at 6K (bar length = score out of 100; FP16 = InfiniPot-V reproduced).

Table 2. MLVU per-task accuracy (6K, Pool)
KV	Topic	Anomaly	Plot	Needle	Ego	AO	AC	Avg
FP16	87.8	70.0	72.7	78.0	63.9	52.9	35.0	65.8
INT4 KIVI	86.7	68.5	74.2	80.0	63.9	55.6	34.0	66.1

Bold marks the better value per task; ties are both bold.

Table 3. KV bytes per token per head
KV	Bytes	Breakdown
FP16	512 B	—
INT4 KIVI	164 B	128 data + 16 V scale/zero + 16 K scale/zero + 4 pos

About 1/3.1 the FP16 size. K per channel, V per token, group size 32, last 128 tokens FP16.

FP16512 B

INT4 KIVI164 B

References

† InfiniPot-V paper (full citation: [TBD]).