# Qwen2.5-VL video captioning, native on Apple Silicon with MLX

> Replicating the Marlin-2B temporal-captioning paradigm on a Mac using MLX: sub-second visual grounding, entirely inside unified memory, no CUDA and no upload.

By Himanshu Maurya · 2026-05-12 · AI · MLX · 2 min
Canonical: https://codingstark.com/blog/qwen-vl-apple-silicon/

Cloud vision inference is fine until you're calling it thousands of times. This project replicates the NemoStation/Marlin-2B temporal-captioning paradigm **natively on Apple Silicon with MLX** — ultra-fast video captioning and sub-second visual grounding, entirely inside the Mac's unified memory.

### Why unified memory changes the maths

On a discrete GPU you pay a tax every time weights and activations cross the PCIe bus. On Apple Silicon the CPU and GPU share the *same* physical memory, so a 2B VLM that would thrash a small VRAM budget simply… fits, and there's no host↔device copy in the hot loop. That single fact is why so much local inference suddenly makes sense on a Mac.

### The hard parts were sampling and the text encoder

Most of the porting effort wasn't the transformer blocks — MLX handles those. It was:

- **Temporal frame sampling** — captioning quality lives or dies on *which* frames you feed. Uniform sampling misses the event; a cheap scene-change heuristic keeps the informative frames.
- **The text encoder / tokenizer parity** — matching the reference tokenization exactly, so embeddings line up with the vision tower.

```python
frames = sample_keyframes(video, target=8)   # scene-aware, not uniform
feats  = vision_tower(frames)                 # stays in unified memory
caption = decoder.generate(feats, prompt)     # streamed, sub-second first token
```

Once frames are sampled sensibly, the model captions video in near real time on an M-series laptop — offline, private, and free per call. For anyone building on-device AI, MLX has quietly become the reason the Mac is a serious inference target.