v0.31.6: Key points on empty frames, zero-shot Transformers results, and drawing on grayscale and palette scenes

A fixes-only patch: fewer crashes on empty frames, more Transformers models that load, and annotators that draw on any scene.

• KeyPoints filters stop raising TypeError on video frames with no people.
• Grounding DINO and RF-DETR segmentation results load, and OWLv2 detections get real class names.
• DetectionsSmoother keeps unconfirmed tracks as separate boxes instead of one averaged box.
• Labels and images draw on grayscale, BGRA and Pillow palette scenes.
• Integer boxes no longer wrap around, and narrow-integer mask shifts no longer crash.

It's a drop-in upgrade, but check the Migration guide if you use ViTPose class IDs, YOLO-NAS confidences or compare stored outputs.

✨ Spotlights / highlights

Key points survive empty frames

sv.KeyPoints.empty() now holds empty class_id, keypoint_confidence and detection_confidence arrays instead of None, and selecting key points from it returns an empty result. Pose connectors such as sv.KeyPoints.from_ultralytics return it for every frame without people, so the documented filters raised TypeError mid-video. ViTPose poses from sv.KeyPoints.from_transformers now all get class ID 0 instead of 0..N-1, and sv.KeyPoints.from_yolo_nas keeps each pose score as detection_confidence, so with_nms works on YOLO-NAS poses. (#2911, #2920, #2914, #2909)

import supervision as sv

key_points = sv.KeyPoints.empty()  # what pose connectors return for an empty frame
len(key_points[key_points.detection_confidence > 0.5])
# 0.31.6: 0  -> correct, an empty frame has nothing to filter
# 0.31.5: TypeError: '>' not supported between instances of 'NoneType' and 'float'
#         -> bug, the confidence arrays were None
key_points[:, [0, 1]].xy.shape
# 0.31.6: (0, 0, 2)  -> correct, no poses so nothing to select
# 0.31.5: IndexError  -> bug, selection crashed on an empty frame

Zero-shot and RF-DETR results from Transformers

sv.Detections.from_transformers now reads results of post_process_grounded_object_detection. Grounding DINO results used to raise AttributeError: 'list' object has no attribute 'cpu'; they now load with the matched phrases as class names and class_id=None. OWL-ViT, OWLv2 and OmDet-Turbo keep the text query index as class_id and take class names from text_labels; id2label=model.config.id2label used to name them LABEL_0, LABEL_1. RF-DETR instance segmentation results requested with return_binary_maps=True load too. If a result has no recognized fields, the error now points to the official rfdetr package, whose predict() returns sv.Detections directly. (#2913, #2912, #2921)

import supervision as sv

# results = processor.post_process_grounded_object_detection(...)[0]  # Grounding DINO
detections = sv.Detections.from_transformers(results)
detections.data["class_name"]  # matched phrases, e.g. ["a cat", "a dog"]
detections.class_id  # None

DetectionsSmoother keeps unconfirmed tracks apart

Detections with a negative tracker_id, which some trackers use for tracks they haven't confirmed yet, now pass through sv.DetectionsSmoother unchanged. They used to share one history and come back as a single averaged box. Frames where only some tracks keep oriented-box corners no longer raise a merge error. (#2926, #2923)

import numpy as np
import supervision as sv

smoother = sv.DetectionsSmoother()
detections = sv.Detections(
    xyxy=np.array([[0, 0, 10, 10], [100, 0, 110, 10]], dtype=float),
    tracker_id=np.array([-1, -1]),  # not confirmed by the tracker yet
)
smoother.update_with_detections(detections).xyxy.tolist()
# 0.31.6: [[0.0, 0.0, 10.0, 10.0], [100.0, 0.0, 110.0, 10.0]]
#         -> correct, two objects stay two boxes
# 0.31.5: [[50.0, 0.0, 60.0, 10.0]]
#         -> bug, both were averaged into one phantom box

Drawing on grayscale, BGRA and palette scenes

sv.RichLabelAnnotator annotates grayscale scenes, shaped (H, W) or (H, W, 1) or given as a Pillow grayscale image, and sv.draw_image draws on grayscale and BGRA scenes. Boxes and labels on Pillow palette images now come out in the colors you asked for, not black. Labels such as 0 or False show up as text instead of empty boxes. (#2916, #2906, #2907, #2892)

import numpy as np
import supervision as sv

scene = np.zeros((100, 100), dtype=np.uint8)  # grayscale frame
detections = sv.Detections(
    xyxy=np.array([[10.0, 10.0, 60.0, 60.0]]), class_id=np.array([0])
)
sv.RichLabelAnnotator().annotate(scene, detections, labels=["car"]).shape
# 0.31.6: (100, 100)  -> correct, grayscale scene annotated in place
# 0.31.5: TypeError: color must be int or single-element tuple
#         -> bug, grayscale scenes were not supported

🔄 Migration guide

Nothing was removed, renamed or deprecated, but a few fixes change what existing code returns:

• Different values: ViTPose poses from KeyPoints.from_transformers all get class_id 0; use np.arange(len(key_points)) if you used it as a per-person index. (#2914) KeyPoints.from_yolo_nas(...).as_detections().confidence reports the model score instead of the mean key point confidence. (#2909) OWL-ViT, OWLv2 and OmDet-Turbo class names come from text_labels, even when you pass id2label. (#2913) xywh_to_xyxy returns int64 for integer input, or float64 when a corner exceeds the int64 range. (#2922)
• Different rows and shapes: DetectionsSmoother returns unconfirmed detections as their own rows, after the smoothed tracks. (#2926) KeyPoints.empty() fields are empty arrays, not None. (#2911) With OpenCV, resize_image, scale_image and letterbox_image keep (H, W, 1) input as (H, W, 1). (#2905)
• Different counts, colors and labels: PolygonZone no longer counts anchors in (-1, 0). (#2924) ColorPalette.from_matplotlib rounds colors like to_hex, so some move by one level. (#2903) Labels such as 0 or False, drawn as empty boxes before, now show their text. (#2892)
• Without OpenCV: mask_to_polygons returns polygons in OpenCV's RETR_TREE order (#2900), image writes save most non-uint8 images instead of failing, though multi-channel non-uint8 TIFFs still fail (#2901), and HeatMapAnnotator colors move closer to OpenCV (#2904). Regenerate stored references made with the NumPy fallback.
• New error: drawing on a Pillow palette image whose palette is full and has no opaque entry raises ValueError instead of drawing wrong colors. (#2907)

📝 Notable changes

🌱 Changed

• Detections.from_transformers suggests the official rfdetr package when a result has no recognized fields, and its docs recommend rfdetr for RF-DETR models. (#2921)

🔧 Fixed

Key points

• KeyPoints.empty() holds empty class_id, keypoint_confidence and detection_confidence arrays instead of None, so confidence filters no longer raise TypeError on frames without poses. (#2911)
• Selecting key points from an empty result, as in key_points[:, [0, 1]], returns an empty result instead of raising IndexError. (#2920)
• KeyPoints.from_transformers gives every ViTPose pose class ID 0; with the old 0..N-1 IDs, keypoint mAP against person targets scored too low and per-class EdgeAnnotator edges raised ValueError. (#2914)
• KeyPoints.from_yolo_nas keeps each pose score as detection_confidence, so with_nms no longer raises ValueError on YOLO-NAS poses. (#2909)

Model connectors

• Detections.from_transformers reads zero-shot results from post_process_grounded_object_detection: Grounding DINO no longer raises AttributeError, and OWL-ViT, OWLv2 and OmDet-Turbo take class names from text_labels. (#2913)
• Detections.from_transformers reads RF-DETR instance segmentation results with return_binary_maps=True, pairing binary maps with segments_info by position; an empty (0, H, W) stack gives empty masks. (#2912)
• Classifications.from_clip and Classifications.from_timm accept bfloat16 outputs, including mixed-precision inference, and return float32 probabilities for them. (#2894)

Tracking, zones and box math

• DetectionsSmoother returns detections with a negative tracker_id unchanged instead of averaging them into one box; when those rows carry oriented corners, confirmed tracks without corner history get their smoothed axis-aligned bounds as corners. (#2926)
• DetectionsSmoother returns all tracked boxes when only some smoothed tracks keep oriented corners, using smoothed axis-aligned bounds instead of raising a merge error. (#2923)
• PolygonZone excludes negative subpixel anchors before rounding, so an anchor just outside the top or left edge is no longer counted as pixel zero. (#2924)
• xywh_to_xyxy promotes integer coordinates before adding widths and heights: a uint8 box [200, 20, 100, 40] becomes [200, 20, 300, 60] instead of wrapping its right edge to 44. (#2922)
• move_masks computes slice bounds without integer overflow, so narrow integer offsets work on larger frames and no-overlap shifts with narrow or unsigned integer offsets return empty masks instead of raising. (#2918)
• JSONSink keeps an exception raised inside a with block when writing or closing the file also fails; the cleanup error is logged. (#2902)

Annotators and colors

• RichLabelAnnotator annotates grayscale scenes, (H, W), (H, W, 1) or Pillow grayscale modes, instead of raising TypeError or ValueError. (#2916)
• Annotators draw the requested colors on Pillow palette images (P, PA) instead of other colors or black. (#2907)
• draw_image draws on grayscale and BGRA scenes instead of raising a NumPy broadcast error. (#2906)
• LabelAnnotator and RichLabelAnnotator draw labels such as 0, np.int64(0), 0.0 and False as text instead of empty background boxes. (#2892)
• ColorPalette.from_matplotlib rounds colors to 8-bit values as Matplotlib's to_hex does instead of truncating them. (#2903)

Images and the NumPy fallback without OpenCV

• resize_image, scale_image and letterbox_image return (H, W, 1) for (H, W, 1) input on OpenCV installs too, and tint_image and grayscale_image accept (H, W, 1) images. (#2905)
• Image writes without OpenCV, such as ImageSink.save_image and dataset exports, keep 16-bit PNGs and save a saturated 8-bit copy for JPEG, BMP and WebP, instead of raising TypeError or returning False. (#2901)
• mask_to_polygons lists polygons in OpenCV's RETR_TREE order without OpenCV too, and one- and two-pixel contours are int32. (#2900)
• HeatMapAnnotator colors without OpenCV follow OpenCV's float32 HSV-to-BGR arithmetic instead of coming out one level off. (#2904)
• create_batches yields each full batch before consuming the next item, so stopping early leaves the rest of the source intact. (#2893)

📚 Docs

• New how-to section on attaching tracker IDs back to key points in Track Objects. (#2896)

🏆 Contributors

Welcome to our first-time contributors Mohammad Sohail Shaikh, Vlad Voropaev, Ahmet Mert Özacar and peterolkhov!

• Oke Iyanuoluwa Enoch (@Iyanuoluwa007, LinkedIn) – drawing on grayscale, BGRA and Pillow palette scenes, selecting key points from empty results, and OpenCV-matching contours, image writes and HSV conversion in the NumPy fallback.
• Mohammad Sohail Shaikh (@sohailshk, LinkedIn) – kept YOLO-NAS pose scores, gave KeyPoints.empty() empty arrays, and kept unconfirmed tracks out of DetectionsSmoother averaging.
• NIKHIL (@Nikhi00718, LinkedIn) – loaded zero-shot and RF-DETR segmentation results in from_transformers and fixed ViTPose class IDs.
• Vlad Voropaev (@voropaevv, LinkedIn) – fixed DetectionsSmoother on mixed oriented-box tracks and PolygonZone edge anchors.
• Mohammad Hijjawi (@MohammadHijjawi97, LinkedIn) – drew falsy labels as text and rounded Matplotlib palette colors like to_hex.
• A Aswanth Raj (@aswanth-07, LinkedIn) – accepted bfloat16 classification outputs and fixed overflow in move_masks.
• Devulapalli Naga Sri Vaishnavi (@Vaishnavi220506, LinkedIn) – kept with block errors in JSONSink and made create_batches stop consuming early.
• peterolkhov (@peterolkhov) – fixed integer overflow in xywh_to_xyxy.
• Kari Pikkarainen (@kari-pikkarainen, LinkedIn) – wrote the how-to on attaching tracker IDs to key points and corrected the contributing guide.
• Ahmet Mert Özacar (@AhmetMert01) – type-checked the docstring example tests.
• Jirka Borovec (@Borda, LinkedIn) – pointed from_transformers users to rfdetr and prepared the release.

Automated contributions: dependabot, pre-commit.ci

Full changelog: 0.31.5...0.31.6
Co-authored with /oss:release.