Transformers 5.20.0 Release Adds Muse Spark Open, Ling 3.0, MiniCPM-V 4.7 and UnlimitedOcr

It is a multimodal mixture-of-experts model from Meta that takes text, images, audio and video as input and generates text. It has 816B total parameters (42B active per token), a context window of up to 1,048,576 tokens shared across modalities, and is trained for agentic coding tasks.

UnlimitedOcr builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention, keeping the KV cache constant during decoding so it can transcribe dozens of pages in a single forward pass. It supports gundam mode (640x640 tiles with dynamic cropping) and base mode (a single 1024x1024 view), and outputs region detections with bounding boxes.

The upgrade is worth evaluating because it adds strong multimodal models and improves distributed parallelism, but it carries breaking changes in cache_utils.py, SubConfigSpec and LUKE token handling, so compatibility should be tested in a staging environment before production rollout.

The minimum is 0.17.2, which enables use_fallback=KernelizeFallback.ALL so a failed kernel load falls back to the original implementation instead of raising an error.