Merge branch 'develop' into dfreilich/add-icon-annotator

This commit is contained in:
LinasKo committed 2024-08-15 11:38:13 +03:00
commit cfd079bbcb
158 files changed
+23506 -4763

No files matched your search

+2 -2
View File
@@ -19,13 +19,13 @@ as an extra within the Supervision package.
```
<div class="md-typeset">
<h2>download_assets</h2>
<h2><a href="#supervision.assets.downloader.download_assets.download_assets">download_assets</a></h2>
</div>
:::supervision.assets.downloader.download_assets
<div class="md-typeset">
<h2>VideoAssets</h2>
<h2><a href="#supervision.assets.downloader.download_assets.VideoAssets">VideoAssets</a></h2>
</div>
:::supervision.assets.list.VideoAssets
+376 -5
View File
@@ -1,6 +1,380 @@
### 0.22.0 <small>Jul 12, 2024</small>
- Added [#1326](https://github.com/roboflow/supervision/pull/1326): [`sv.DetectionsDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset) and [`sv.ClassificationDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.ClassificationDataset) allowing to load the images into memory only when necessary (lazy loading).
!!! failure "Deprecated"
Constructing `DetectionDataset` with parameter `images` as `Dict[str, np.ndarray]` is deprecated and will be removed in `supervision-0.26.0`. Please pass a list of paths `List[str]` instead.
!!! failure "Deprecated"
The `DetectionDataset.images` property is deprecated and will be removed in `supervision-0.26.0`. Please loop over images with `for path, image, annotation in dataset:`, as that does not require loading all images into memory.
```python
import roboflow
from roboflow import Roboflow
import supervision as sv
roboflow.login()
rf = Roboflow()
project = rf.workspace(<WORKSPACE_ID>).project(<PROJECT_ID>)
dataset = project.version(<PROJECT_VERSION>).download("coco")
ds_train = sv.DetectionDataset.from_coco(
images_directory_path=f"{dataset.location}/train",
annotations_path=f"{dataset.location}/train/_annotations.coco.json",
)
path, image, annotation = ds_train[0]
# loads image on demand
for path, image, annotation in ds_train:
# loads image on demand
```
- Added [#1296](https://github.com/roboflow/supervision/pull/1296): [`sv.Detections.from_lmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_lmm) now supports parsing results from the [Florence 2](https://huggingface.co/microsoft/Florence-2-large) model, extending the capability to handle outputs from this Large Multimodal Model (LMM). This includes detailed object detection, OCR with region proposals, segmentation, and more. Find out more in our [Colab notebook](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-finetune-florence-2-on-detection-dataset.ipynb).
- Added [#1232](https://github.com/roboflow/supervision/pull/1232) to support keypoint detection with Mediapipe. Both [legacy](https://colab.research.google.com/github/googlesamples/mediapipe/blob/main/examples/pose_landmarker/python/%5BMediaPipe_Python_Tasks%5D_Pose_Landmarker.ipynb) and [modern](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker/python) pipelines are supported. See [`sv.KeyPoints.from_mediapipe`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_mediapipe) for more.
- Added [#1316](https://github.com/roboflow/supervision/pull/1316): [`sv.KeyPoints.from_mediapipe`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_mediapipe) extended to support FaceMesh from Mediapipe. This enhancement allows for processing both face landmarks from `FaceLandmarker`, and legacy results from `FaceMesh`.
- Added [#1310](https://github.com/roboflow/supervision/pull/1310): [`sv.KeyPoints.from_detectron2`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_detectron2) is a new `KeyPoints` method, adding support for extracting keypoints from the popular [Detectron 2](https://github.com/facebookresearch/detectron2) platform.
- Added [#1300](https://github.com/roboflow/supervision/pull/1300): [`sv.Detections.from_detectron2`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_detectron2) now supports segmentation models detectron2. The resulting masks can be used with [`sv.MaskAnnotator`](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.MaskAnnotator) for displaying annotations.
```python
import supervision as sv
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
import cv2
image = cv2.imread(<SOURCE_IMAGE_PATH>)
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
predictor = DefaultPredictor(cfg)
result = predictor(image)
detections = sv.Detections.from_detectron2(result)
mask_annotator = sv.MaskAnnotator()
annotated_frame = mask_annotator.annotate(scene=image.copy(), detections=detections)
```
- Added [#1277](https://github.com/roboflow/supervision/pull/1277): if you provide a font that supports symbols of a language, [`sv.RichLabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator.annotate) will draw them on your images.
- Various other annotators have been revised to ensure proper in-place functionality when used with `numpy` arrays. Additionally, we fixed a bug where `sv.ColorAnnotator` was filling boxes with solid color when used in-place.
```python
import cv2
import supervision as sv
import
image = cv2.imread(<SOURCE_IMAGE_PATH>)
model = get_model(model_id="yolov8n-640")
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
rich_label_annotator = sv.RichLabelAnnotator(font_path=<TTF_FONT_PATH>)
annotated_image = rich_label_annotator.annotate(scene=image.copy(), detections=detections)
```
- Added [#1227](https://github.com/roboflow/supervision/pull/1227): Added support for loading Oriented Bounding Boxes dataset in YOLO format.
```python
import supervision as sv
train_ds = sv.DetectionDataset.from_yolo(
images_directory_path="/content/dataset/train/images",
annotations_directory_path="/content/dataset/train/labels",
data_yaml_path="/content/dataset/data.yaml",
is_obb=True
)
_, image, detections in train_ds[0]
obb_annotator = OrientedBoxAnnotator()
annotated_image = obb_annotator.annotate(scene=image.copy(), detections=detections)
```
- Fixed [#1312](https://github.com/roboflow/supervision/pull/1312): Fixed [`CropAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.TraceAnnotator.annotate).
!!! failure "Removed"
`BoxAnnotator` was removed, however `BoundingBoxAnnotator` has been renamed to `BoxAnnotator`. Use a combination of [`BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator) and [`LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) to simulate old `BoundingBox` behavior.
!!! failure "Deprecated"
The name `BoundingBoxAnnotator` has been deprecated and will be removed in `supervision-0.26.0`. It has been renamed to [`BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator).
- Added [#975](https://github.com/roboflow/supervision/pull/975) 📝 New Cookbooks: serialize detections into [json](https://github.com/roboflow/supervision/blob/de896189b83a1f9434c0a37dd9192ee00d2a1283/docs/notebooks/serialise-detections-to-json.ipynb) and [csv](https://github.com/roboflow/supervision/blob/de896189b83a1f9434c0a37dd9192ee00d2a1283/docs/notebooks/serialise-detections-to-csv.ipynb).
- Added [#1290](https://github.com/roboflow/supervision/pull/1290): Mostly an internal change, our file utility function now support both `str` and `pathlib` paths.
- Added [#1340](https://github.com/roboflow/supervision/pull/1340): Two new methods for converting between bounding box formats - [`xywh_to_xyxy`](https://supervision.roboflow.com/latest/detection/utils/#supervision.detection.utils.xywh_to_xyxy) and [`xcycwh_to_xyxy`](https://supervision.roboflow.com/latest/detection/utils/#supervision.detection.utils.xcycwh_to_xyxy)
!!! failure "Removed"
`from_roboflow` method has been removed due to deprecation. Use [from_inference](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_inference) instead.
!!! failure "Removed"
`Color.white()` has been removed due to deprecation. Use `color.WHITE` instead.
!!! failure "Removed"
`Color.black()` has been removed due to deprecation. Use `color.BLACK` instead.
!!! failure "Removed"
`Color.red()` has been removed due to deprecation. Use `color.RED` instead.
!!! failure "Removed"
`Color.green()` has been removed due to deprecation. Use `color.GREEN` instead.
!!! failure "Removed"
`Color.blue()` has been removed due to deprecation. Use `color.BLUE` instead.
!!! failure "Removed"
`ColorPalette.default()` has been removed due to deprecation. Use [ColorPalette.DEFAULT](https://supervision.roboflow.com/latest/utils/draw/#supervision.draw.color.ColorPalette.DEFAULT) instead.
!!! failure "Removed"
`FPSMonitor.__call__` has been removed due to deprecation. Use the attribute [FPSMonitor.fps](https://supervision.roboflow.com/latest/utils/video/#supervision.utils.video.FPSMonitor.fps) instead.
### 0.21.0 <small>Jun 5, 2024</small>
- Added [#500](https://github.com/roboflow/supervision/pull/500): [`sv.Detections.with_nmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.with_nmm) to perform non-maximum merging on the current set of object detections.
- Added [#1221](https://github.com/roboflow/supervision/pull/1221): [`sv.Detections.from_lmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_lmm) allowing to parse Large Multimodal Model (LMM) text result into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/) object. For now `from_lmm` supports only [PaliGemma](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-finetune-paligemma-on-detection-dataset.ipynb) result parsing.
```python
import supervision as sv
paligemma_result = "<loc0256><loc0256><loc0768><loc0768> cat"
detections = sv.Detections.from_lmm(
sv.LMM.PALIGEMMA,
paligemma_result,
resolution_wh=(1000, 1000),
classes=['cat', 'dog']
)
detections.xyxy
# array([[250., 250., 750., 750.]])
detections.class_id
# array([0])
```
- Added [#1236](https://github.com/roboflow/supervision/pull/1236): [`sv.VertexLabelAnnotator`](https://supervision.roboflow.com/latest/keypoint/annotators/#supervision.keypoint.annotators.EdgeAnnotator.annotate) allowing to annotate every vertex of a keypoint skeleton with custom text and color.
```python
import supervision as sv
image = ...
key_points = sv.KeyPoints(...)
edge_annotator = sv.EdgeAnnotator(
color=sv.Color.GREEN,
thickness=5
)
annotated_frame = edge_annotator.annotate(
scene=image.copy(),
key_points=key_points
)
```
- Added [#1147](https://github.com/roboflow/supervision/pull/1147): [`sv.KeyPoints.from_inference`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints.from_inference) allowing to create [`sv.KeyPoints`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints) from [Inference](https://github.com/roboflow/inference) result.
- Added [#1138](https://github.com/roboflow/supervision/pull/1138): [`sv.KeyPoints.from_yolo_nas`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints.from_yolo_nas) allowing to create [`sv.KeyPoints`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints) from [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) result.
- Added [#1163](https://github.com/roboflow/supervision/pull/1163): [`sv.mask_to_rle`](https://supervision.roboflow.com/develop/datasets/utils/#supervision.dataset.utils.rle_to_mask) and [`sv.rle_to_mask`](https://supervision.roboflow.com/develop/datasets/utils/#supervision.dataset.utils.rle_to_mask) allowing for easy conversion between mask and rle formats.
- Changed [#1236](https://github.com/roboflow/supervision/pull/1236): [`sv.InferenceSlicer`](https://supervision.roboflow.com/develop/detection/tools/inference_slicer/) allowing to select overlap filtering strategy (`NONE`, `NON_MAX_SUPPRESSION` and `NON_MAX_MERGE`).
- Changed [#1178](https://github.com/roboflow/supervision/pull/1178): [`sv.InferenceSlicer`](https://supervision.roboflow.com/develop/detection/tools/inference_slicer/) adding instance segmentation model support.
```python
import cv2
import numpy as np
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8x-seg-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
results = model.infer(image_slice)[0]
return sv.Detections.from_inference(results)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
- Changed [#1228](https://github.com/roboflow/supervision/pull/1228): [`sv.LineZone`](https://supervision.roboflow.com/develop/detection/tools/line_zone/) making it 10-20 times faster, depending on the use case.
- Changed [#1163](https://github.com/roboflow/supervision/pull/1163): [`sv.DetectionDataset.from_coco`](https://supervision.roboflow.com/develop/datasets/core/#supervision.dataset.core.DetectionDataset.from_coco) and [`sv.DetectionDataset.as_coco`](https://supervision.roboflow.com/develop/datasets/core/#supervision.dataset.core.DetectionDataset.as_coco) adding support for run-length encoding (RLE) mask format.
### 0.20.0 <small>April 24, 2024</small>
- Added [#1128](https://github.com/roboflow/supervision/pull/1128): [`sv.KeyPoints`](/0.20.0/keypoint/core/#supervision.keypoint.core.KeyPoints) to provide initial support for pose estimation and broader keypoint detection models.
- Added [#1128](https://github.com/roboflow/supervision/pull/1128): [`sv.EdgeAnnotator`](/0.20.0/keypoint/annotators/#supervision.keypoint.annotators.EdgeAnnotator) and [`sv.VertexAnnotator`](/0.20.0/keypoint/annotators/#supervision.keypoint.annotators.VertexAnnotator) to enable rendering of results from keypoint detection models.
```python
import cv2
import supervision as sv
from ultralytics import YOLO
image = cv2.imread(<SOURCE_IMAGE_PATH>)
model = YOLO('yolov8l-pose')
result = model(image, verbose=False)[0]
keypoints = sv.KeyPoints.from_ultralytics(result)
edge_annotators = sv.EdgeAnnotator(color=sv.Color.GREEN, thickness=5)
annotated_image = edge_annotators.annotate(image.copy(), keypoints)
```
- Changed [#1037](https://github.com/roboflow/supervision/pull/1037): [`sv.LabelAnnotator`](/0.20.0/annotators/#supervision.annotators.core.LabelAnnotator) by adding an additional `corner_radius` argument that allows for rounding the corners of the bounding box.
- Changed [#1109](https://github.com/roboflow/supervision/pull/1109): [`sv.PolygonZone`](/0.20.0/detection/tools/polygon_zone/#supervision.detection.tools.polygon_zone.PolygonZone) such that the `frame_resolution_wh` argument is no longer required to initialize `sv.PolygonZone`.
!!! failure "Deprecated"
The `frame_resolution_wh` parameter in `sv.PolygonZone` is deprecated and will be removed in `supervision-0.24.0`.
- Changed [#1084](https://github.com/roboflow/supervision/pull/1084): [`sv.get_polygon_center`](/0.20.0/utils/geometry/#supervision.geometry.core.utils.get_polygon_center) to calculate a more accurate polygon centroid.
- Changed [#1069](https://github.com/roboflow/supervision/pull/1069): [`sv.Detections.from_transformers`](/0.20.0/detection/core/#supervision.detection.core.Detections.from_transformers) by adding support for Transformers segmentation models and extract class names values.
```python
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForSegmentation
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50-panoptic")
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_segmentation(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results, id2label=model.config.id2label)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER)
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
- Fixed [#787](https://github.com/roboflow/supervision/pull/787): [`sv.ByteTrack.update_with_detections`](/0.20.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack.update_with_detections) which was removing segmentation masks while tracking. Now, `ByteTrack` can be used alongside segmentation models.
### 0.19.0 <small>March 15, 2024</small>
- Added [#818](https://github.com/roboflow/supervision/pull/818): [`sv.CSVSink`](/0.19.0/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) allowing for the straightforward saving of image, video, or stream inference results in a `.csv` file.
```python
import supervision as sv
from ultralytics import YOLO
model = YOLO(<SOURCE_MODEL_PATH>)
csv_sink = sv.CSVSink(<RESULT_CSV_FILE_PATH>)
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with csv_sink:
for frame in frames_generator:
result = model(frame)[0]
detections = sv.Detections.from_ultralytics(result)
csv_sink.append(detections, custom_data={<CUSTOM_LABEL>:<CUSTOM_DATA>})
```
- Added [#819](https://github.com/roboflow/supervision/pull/819): [`sv.JSONSink`](/0.19.0/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink) allowing for the straightforward saving of image, video, or stream inference results in a `.json` file.
```python
import supervision as sv
from ultralytics import YOLO
model = YOLO(<SOURCE_MODEL_PATH>)
json_sink = sv.JSONSink(<RESULT_JSON_FILE_PATH>)
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with json_sink:
for frame in frames_generator:
result = model(frame)[0]
detections = sv.Detections.from_ultralytics(result)
json_sink.append(detections, custom_data={<CUSTOM_LABEL>:<CUSTOM_DATA>})
```
- Added [#847](https://github.com/roboflow/supervision/pull/847): [`sv.mask_iou_batch`](/0.19.0/detection/utils/#supervision.detection.utils.mask_iou_batch) allowing to compute Intersection over Union (IoU) of two sets of masks.
- Added [#847](https://github.com/roboflow/supervision/pull/847): [`sv.mask_non_max_suppression`](/0.19.0/detection/utils/#supervision.detection.utils.mask_non_max_suppression) allowing to perform Non-Maximum Suppression (NMS) on segmentation predictions.
- Added [#888](https://github.com/roboflow/supervision/pull/888): [`sv.CropAnnotator`](/0.19.0/annotators/#supervision.annotators.core.CropAnnotator) allowing users to annotate the scene with scaled-up crops of detections.
```python
import cv2
import supervision as sv
from inference import get_model
image = cv2.imread(<SOURCE_IMAGE_PATH>)
model = get_model(model_id="yolov8n-640")
result = model.infer(image)[0]
detections = sv.Detections.from_inference(result)
crop_annotator = sv.CropAnnotator()
annotated_frame = crop_annotator.annotate(
scene=image.copy(),
detections=detections
)
```
- Changed [#827](https://github.com/roboflow/supervision/pull/827): [`sv.ByteTrack.reset`](/0.19.0/tracking/#supervision.tracking.ByteTrack.reset) allowing users to clear trackers state, enabling the processing of multiple video files in sequence.
- Changed [#802](https://github.com/roboflow/supervision/pull/802): [`sv.LineZoneAnnotator`](/0.19.0/detection/tools/line_zone/#supervision.detection.line_zone.LineZone) allowing to hide in/out count using `display_in_count` and `display_out_count` properties.
- Changed [#787](https://github.com/roboflow/supervision/pull/787): [`sv.ByteTrack`](/0.19.0/tracking/#supervision.tracking.ByteTrack) input arguments and docstrings updated to improve readability and ease of use.
!!! failure "Deprecated"
The `track_buffer`, `track_thresh`, and `match_thresh` parameters in `sv.ByterTrack` are deprecated and will be removed in `supervision-0.23.0`. Use `lost_track_buffer,` `track_activation_threshold`, and `minimum_matching_threshold` instead.
- Changed [#910](https://github.com/roboflow/supervision/pull/910): [`sv.PolygonZone`](/0.19.0/detection/tools/polygon_zone/#supervision.detection.tools.polygon_zone.PolygonZone) to now accept a list of specific box anchors that must be in zone for a detection to be counted.
!!! failure "Deprecated"
The `triggering_position ` parameter in `sv.PolygonZone` is deprecated and will be removed in `supervision-0.23.0`. Use `triggering_anchors` instead.
- Changed [#875](https://github.com/roboflow/supervision/pull/875): annotators adding support for Pillow images. All supervision Annotators can now accept an image as either a numpy array or a Pillow Image. They automatically detect its type, draw annotations, and return the output in the same format as the input.
- Fixed [#944](https://github.com/roboflow/supervision/pull/944): [`sv.DetectionsSmoother`](/0.19.0/detection/tools/smoother/#supervision.detection.tools.smoother.DetectionsSmoother) removing `tracking_id` from `sv.Detections`.
### 0.18.0 <small>January 25, 2024</small>
- Added [#633](https://github.com/roboflow/supervision/pull/720): [`sv.PercentageBarAnnotator`](/0.18.0/annotators/#percentagebarannotator) allowing to annotate images and videos with percentage values representing confidence or other custom property.
- Added [#720](https://github.com/roboflow/supervision/pull/720): [`sv.PercentageBarAnnotator`](/0.18.0/annotators/#percentagebarannotator) allowing to annotate images and videos with percentage values representing confidence or other custom property.
```python
>>> import supervision as sv
@@ -58,7 +432,6 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
`sv.ColorPalette.default()` is deprecated and will be removed in `supervision-0.22.0`. Use `sv.ColorPalette.DEFAULT` instead.
- Changed [#769](https://github.com/roboflow/supervision/pull/769): [`sv.ColorPalette.DEFAULT`](/0.18.0/draw/color/#colorpalette) value, giving users a more extensive set of annotation colors.
- Changed [#677](https://github.com/roboflow/supervision/pull/677): `sv.Detections.from_roboflow` to [`sv.Detections.from_inference`](/0.18.0/detection/core/#supervision.detection.core.Detections.from_inference) streamlining its functionality to be compatible with both the both [inference](https://github.com/roboflow/inference) pip package and the Robloflow [hosted API](https://docs.roboflow.com/deploy/hosted-api).
@@ -67,7 +440,6 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
`Detections.from_roboflow()` is deprecated and will be removed in `supervision-0.22.0`. Use `Detections.from_inference` instead.
- Fixed [#735](https://github.com/roboflow/supervision/pull/735): [`sv.LineZone`](/0.18.0/detection/tools/line_zone/#linezone) functionality to accurately update the counter when an object crosses a line from any direction, including from the side. This enhancement enables more precise tracking and analytics, such as calculating individual in/out counts for each lane on the road.
### 0.17.0 <small>December 06, 2023</small>
@@ -160,13 +532,12 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
- Fixed [#477](https://github.com/roboflow/supervision/pull/477): Poetry env definition allowing proper local installation.
- Fixed [#430](https://github.com/roboflow/supervision/pull/430): [`sv.ByteTrack`](/0.16.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack) to return `np.array([], dtype=int)` when `svDetections` is empty.
- Fixed [#430](https://github.com/roboflow/supervision/pull/430): [`sv.ByteTrack`](/0.16.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack) to return `np.array([], dtype=int)` when `svDetections` is empty.
!!! failure "Deprecated"
`sv.Detections.from_yolov8` and `sv.Classifications.from_yolov8` as those are now replaced by [`sv.Detections.from_ultralytics`](/0.16.0/detection/core/#supervision.detection.core.Detections.from_ultralytics) and [`sv.Classifications.from_ultralytics`](/0.16.0/classification/core/#supervision.classification.core.Classifications.from_ultralytics).
### 0.15.0 <small>October 5, 2023</small>
- Added [#170](https://github.com/roboflow/supervision/pull/170): [`sv.BoundingBoxAnnotator`](/0.15.0/annotators/#supervision.annotators.core.BoundingBoxAnnotator) allowing to annotate images and videos with bounding boxes.
-1
View File
@@ -1,7 +1,6 @@
---
template: cookbooks.html
comments: true
status: new
hide:
- navigation
- toc
@@ -1,5 +1,6 @@
---
comments: true
status: new
---
# Datasets
+17
View File
@@ -0,0 +1,17 @@
---
comments: true
---
# Datasets Utils
<div class="md-typeset">
<h2><a href="#supervision.dataset.utils.rle_to_mask">rle_to_mask</a></h2>
</div>
:::supervision.dataset.utils.rle_to_mask
<div class="md-typeset">
<h2><a href="#supervision.dataset.utils.mask_to_rle">mask_to_rle</a></h2>
</div>
:::supervision.dataset.utils.mask_to_rle
+21 -10
View File
@@ -3,14 +3,25 @@ comments: true
status: deprecated
---
These features are phased out due to better alternatives or potential issues in future versions. Deprecated functionalities are supported for **three subsequent releases**, providing time for users to transition to updated methods.
# Deprecated
- [`Detections.from_froboflow`](detection/core.md/#supervision.detection.core.Detections.from_roboflow) is deprecated and will be removed in `supervision-0.22.0`. Use [`Detections.from_inference`](detection/core.md/#supervision.detection.core.Detections.from_inference) instead.
- `Color.white()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.WHITE` instead.
- `Color.black()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.BLACK` instead.
- `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
- `Color.green()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.GREEN` instead.
- `Color.blue()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.BLUE` instead.
- [`ColorPalette.default()`](draw/color.md/#supervision.draw.color.ColorPalette.default) is deprecated and will be removed in `supervision-0.22.0`. Use [`ColorPalette.DEFAULT`](draw/color.md/#supervision.draw.color.ColorPalette.DEFAULT) instead.
- `BoxAnnotator` is deprecated and will be removed in `supervision-0.22.0`. Use [`BoundingBoxAnnotator`](annotators.md/#supervision.annotators.core.BoundingBoxAnnotator) and [`LabelAnnotator`](annotators.md/#supervision.annotators.core.LabelAnnotator) instead.
- [`FPSMonitor.__call__`](utils/video.md/#supervision.utils.video.FPSMonitor.__call__) is deprecated and will be removed in `supervision-0.22.0`. Use [`FPSMonitor.fps`](utils/video.md/#supervision.utils.video.FPSMonitor.fps) instead.
These features are phased out due to better alternatives or potential issues in future versions. Deprecated functionalities are supported for **five subsequent releases**, providing time for users to transition to updated methods.
- The `track_buffer`, `track_thresh`, and `match_thresh` parameters in [`ByterTrack`](trackers.md/#supervision.tracker.byte_tracker.core.ByteTrack) are deprecated and will be removed in `supervision-0.23.0`. Use `lost_track_buffer,` `track_activation_threshold`, and `minimum_matching_threshold` instead.
- The `triggering_position ` parameter in [`sv.PolygonZone`](detection/tools/polygon_zone.md/#supervision.detection.tools.polygon_zone.PolygonZone) will be removed in `supervision-0.23.0`. Use `triggering_anchors ` instead.
- The `frame_resolution_wh ` parameter in [`sv.PolygonZone`](detection/tools/polygon_zone.md/#supervision.detection.tools.polygon_zone.PolygonZone) will be removed in `supervision-0.24.0`.
- Constructing `DetectionDataset` and `ClassificationDataset` with parameter `images` as `Dict[str, np.ndarray]` will be removed in `supervision-0.26.0`. Please pass a list of paths `List[str]` instead.
- The `DetectionDataset.images` property will be removed in `supervision-0.26.0`. Please loop over images with `for path, image, annotation in dataset:`, as that does not require loading all images into memory.
- `BoundingBoxAnnotator` has been renamed to `BoxAnnotator` after the old implementation of `BoxAnnotator` has been removed. `BoundingBoxAnnotator` will be removed in `supervision-0.26.0`.
# Removed
- [`Detections.from_froboflow`](detection/core.md/#supervision.detection.core.Detections.from_roboflow) is removed as of `supervision-0.22.0`. Use [`Detections.from_inference`](detection/core.md/#supervision.detection.core.Detections.from_inference) instead.
- The method `Color.white()` was removed as of `supervision-0.22.0`. Use the constant `Color.WHITE` instead.
- The method `Color.black()` was removed as of `supervision-0.22.0`. Use the constant `Color.BLACK` instead.
- The method `Color.red()` was removed as of `supervision-0.22.0`. Use the constant `Color.RED` instead.
- The method `Color.green()` was removed as of `supervision-0.22.0`. Use the constant `Color.GREEN` instead.
- The method `Color.blue()` was removed as of `supervision-0.22.0`. Use the constant `Color.BLUE` instead.
- The method [`ColorPalette.default()`](draw/color.md/#supervision.draw.color.ColorPalette.default) was removed as of `supervision-0.22.0`. Use the constant [`ColorPalette.DEFAULT`](draw/color.md/#supervision.draw.color.ColorPalette.DEFAULT) instead.
- `BoxAnnotator` was removed as of `supervision-0.22.0`, however `BoundingBoxAnnotator` was immediately renamed to `BoxAnnotator`. Use [`BoxAnnotator`](detection/annotators.md/#supervision.annotators.core.BoxAnnotator) and [`LabelAnnotator`](detection/annotators.md/#supervision.annotators.core.LabelAnnotator) instead of the old `BoxAnnotator`.
- The method [`FPSMonitor.__call__`](utils/video.md/#supervision.utils.video.FPSMonitor.__call__) was removed as of `supervision-0.22.0`. Use the attribute [`FPSMonitor.fps`](utils/video.md/#supervision.utils.video.FPSMonitor.fps) instead.
@@ -5,7 +5,7 @@ status: new
# Annotators
=== "BoundingBox"
=== "Box"
```python
import supervision as sv
@@ -13,8 +13,8 @@ status: new
image = ...
detections = sv.Detections(...)
bounding_box_annotator = sv.BoundingBoxAnnotator()
annotated_frame = bounding_box_annotator.annotate(
box_annotator = sv.BoxAnnotator()
annotated_frame = box_annotator.annotate(
scene=image.copy(),
detections=detections
)
@@ -260,15 +260,22 @@ status: new
=== "Label"
```python
import supervision as sv
import supervision as sv
image = ...
detections = sv.Detections(...)
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER)
annotated_frame = label_annotator.annotate(
scene=image.copy(),
detections=detections
detections=detections,
labels=labels
)
```
@@ -278,6 +285,52 @@ status: new
</div>
=== "RichLabel"
```python
import supervision as sv
image = ...
detections = sv.Detections(...)
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
rich_label_annotator = sv.RichLabelAnnotator(
font_path="<TTF_FONT_PATH>",
text_position=sv.Position.CENTER
)
annotated_frame = rich_label_annotator.annotate(
scene=image.copy(),
detections=detections,
labels=labels
)
```
<div class="result" markdown>
![label-annotator-example](https://media.roboflow.com/supervision-annotator-examples/label-annotator-example-purple.png){ align=center width="800" }
</div>
=== "Crop"
```python
import supervision as sv
image = ...
detections = sv.Detections(...)
crop_annotator = sv.CropAnnotator()
annotated_frame = crop_annotator.annotate(
scene=image.copy(),
detections=detections
)
```
=== "Blur"
```python
@@ -380,11 +433,32 @@ status: new
</div>
=== "Background Color"
```python
import supervision as sv
image = ...
detections = sv.Detections(...)
background_overlay_annotator = sv.BackgroundOverlayAnnotator()
annotated_frame = background_overlay_annotator.annotate(
scene=image.copy(),
detections=detections
)
```
<div class="result" markdown>
![background-overlay-annotator-example](https://media.roboflow.com/supervision-annotator-examples/background-color-annotator-example-purple.png)
</div>
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.BoundingBoxAnnotator">BoundingBoxAnnotator</a></h2>
<h2><a href="#supervision.annotators.core.BoxAnnotator">BoxAnnotator</a></h2>
</div>
:::supervision.annotators.core.BoundingBoxAnnotator
:::supervision.annotators.core.BoxAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.RoundBoxAnnotator">RoundBoxAnnotator</a></h2>
@@ -470,6 +544,18 @@ status: new
:::supervision.annotators.core.LabelAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.RichLabelAnnotator">RichLabelAnnotator</a></h2>
</div>
:::supervision.annotators.core.RichLabelAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.IconAnnotator">IconAnnotator</a></h2>
</div>
:::supervision.annotators.core.IconAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.BlurAnnotator">BlurAnnotator</a></h2>
</div>
@@ -489,10 +575,16 @@ status: new
:::supervision.annotators.core.TraceAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.IconAnnotator">IconAnnotator</a></h2>
<h2><a href="#supervision.annotators.core.CropAnnotator">CropAnnotator</a></h2>
</div>
:::supervision.annotators.core.IconAnnotator
:::supervision.annotators.core.CropAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.BackgroundOverlayAnnotator">BackgroundOverlayAnnotator</a></h2>
</div>
:::supervision.annotators.core.BackgroundOverlayAnnotator
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.ColorLookup">ColorLookup</a></h2>
+29
View File
@@ -0,0 +1,29 @@
---
comments: true
---
# Double Detection Filter
<div class="md-typeset">
<h2><a href="#supervision.detection.overlap_filter.OverlapFilter">OverlapFilter</a></h2>
</div>
:::supervision.detection.overlap_filter.OverlapFilter
<div class="md-typeset">
<h2><a href="#supervision.detection.overlap_filter.box_non_max_suppression">box_non_max_suppression</a></h2>
</div>
:::supervision.detection.overlap_filter.box_non_max_suppression
<div class="md-typeset">
<h2><a href="#supervision.detection.overlap_filter.mask_non_max_suppression">mask_non_max_suppression</a></h2>
</div>
:::supervision.detection.overlap_filter.mask_non_max_suppression
<div class="md-typeset">
<h2><a href="#supervision.detection.overlap_filter.box_non_max_merge">box_non_max_merge</a></h2>
</div>
:::supervision.detection.overlap_filter.box_non_max_merge
+17
View File
@@ -0,0 +1,17 @@
---
comments: true
---
# Metrics
<div class="md-typeset">
<h2><a href="#supervision.metrics.detection.ConfusionMatrix">ConfusionMatrix</a></h2>
</div>
:::supervision.metrics.detection.ConfusionMatrix
<div class="md-typeset">
<h2><a href="#supervision.metrics.detection.MeanAveragePrecision">MeanAveragePrecision</a></h2>
</div>
:::supervision.metrics.detection.MeanAveragePrecision
+6 -1
View File
@@ -1,6 +1,5 @@
---
comments: true
status: new
---
# Save Detections
@@ -10,3 +9,9 @@ status: new
</div>
:::supervision.detection.tools.csv_sink.CSVSink
<div class="md-typeset">
<h2>JSON Sink</h2>
</div>
:::supervision.detection.tools.json_sink.JSONSink
+52 -21
View File
@@ -1,71 +1,102 @@
---
comments: true
status: new
---
# Detection Utils
<div class="md-typeset">
<h2>box_iou_batch</h2>
<h2><a href="#supervision.detection.utils.box_iou_batch">box_iou_batch</a></h2>
</div>
:::supervision.detection.utils.box_iou_batch
<div class="md-typeset">
<h2>mask_iou_batch</h2>
<h2><a href="#supervision.detection.utils.mask_iou_batch">mask_iou_batch</a></h2>
</div>
:::supervision.detection.utils.mask_iou_batch
<div class="md-typeset">
<h2>box_non_max_suppression</h2>
</div>
:::supervision.detection.utils.box_non_max_suppression
<div class="md-typeset">
<h2>mask_non_max_suppression</h2>
</div>
:::supervision.detection.utils.mask_non_max_suppression
<div class="md-typeset">
<h2>polygon_to_mask</h2>
<h2><a href="#supervision.detection.utils.polygon_to_mask">polygon_to_mask</a></h2>
</div>
:::supervision.detection.utils.polygon_to_mask
<div class="md-typeset">
<h2>mask_to_xyxy</h2>
<h2><a href="#supervision.detection.utils.mask_to_xyxy">mask_to_xyxy</a></h2>
</div>
:::supervision.detection.utils.mask_to_xyxy
<div class="md-typeset">
<h2>mask_to_polygons</h2>
<h2><a href="#supervision.detection.utils.mask_to_polygons">mask_to_polygons</a></h2>
</div>
:::supervision.detection.utils.mask_to_polygons
<div class="md-typeset">
<h2>polygon_to_xyxy</h2>
<h2><a href="#supervision.detection.utils.polygon_to_xyxy">polygon_to_xyxy</a></h2>
</div>
:::supervision.detection.utils.polygon_to_xyxy
<div class="md-typeset">
<h2>filter_polygons_by_area</h2>
<h2><a href="#supervision.detection.utils.filter_polygons_by_area">filter_polygons_by_area</a></h2>
</div>
:::supervision.detection.utils.filter_polygons_by_area
<div class="md-typeset">
<h2>move_boxes</h2>
<h2><a href="#supervision.detection.utils.move_boxes">move_boxes</a></h2>
</div>
:::supervision.detection.utils.move_boxes
<div class="md-typeset">
<h2>scale_boxes</h2>
<h2><a href="#supervision.detection.utils.move_masks">move_masks</a></h2>
</div>
:::supervision.detection.utils.move_masks
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.scale_boxes">scale_boxes</a></h2>
</div>
:::supervision.detection.utils.scale_boxes
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.clip_boxes">clip_boxes</a></h2>
</div>
:::supervision.detection.utils.clip_boxes
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.pad_boxes">pad_boxes</a></h2>
</div>
:::supervision.detection.utils.pad_boxes
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.contains_holes">contains_holes</a></h2>
</div>
:::supervision.detection.utils.xywh_to_xyxy
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.xywh_to_xyxy">xywh_to_xyxy</a></h2>
</div>
:::supervision.detection.utils.xcycwh_to_xyxy
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.xcycwh_to_xyxy">xcycwh_to_xyxy</a></h2>
</div>
:::supervision.detection.utils.contains_holes
<div class="md-typeset">
<h2><a href="#supervision.detection.utils.contains_multiple_segments">contains_multiple_segments</a></h2>
</div>
:::supervision.detection.utils.contains_multiple_segments
-13
View File
@@ -1,13 +0,0 @@
---
comments: true
---
# Color
:::supervision.draw.color.Color
<div class="md-typeset">
<h2>ColorPalette</h2>
</div>
:::supervision.draw.color.ColorPalette
-53
View File
@@ -1,53 +0,0 @@
---
comments: true
---
# Draw Utils
<div class="md-typeset">
<h2>draw_line</h2>
</div>
:::supervision.draw.utils.draw_line
<div class="md-typeset">
<h2>draw_rectangle</h2>
</div>
:::supervision.draw.utils.draw_rectangle
<div class="md-typeset">
<h2>draw_filled_rectangle</h2>
</div>
:::supervision.draw.utils.draw_filled_rectangle
<div class="md-typeset">
<h2>draw_polygon</h2>
</div>
:::supervision.draw.utils.draw_polygon
<div class="md-typeset">
<h2>draw_text</h2>
</div>
:::supervision.draw.utils.draw_text
<div class="md-typeset">
<h2>draw_image</h2>
</div>
:::supervision.draw.utils.draw_image
<div class="md-typeset">
<h2>calculate_dynamic_font_scale</h2>
</div>
:::supervision.draw.utils.calculate_dynamic_text_scale
<div class="md-typeset">
<h2>calculate_dynamic_line_thickness</h2>
</div>
:::supervision.draw.utils.calculate_dynamic_line_thickness
-7
View File
@@ -1,7 +0,0 @@
---
comments: true
---
# Position
:::supervision.geometry.core.Position
+342 -77
View File
@@ -4,17 +4,31 @@ comments: true
# Detect and Annotate
Supervision offers a streamlined solution to effortlessly annotate predictions from a
range of object detection and segmentation models. This guide demonstrates how to
execute inference using the YOLOv8 model with either the
[Inference](https://github.com/roboflow/inference) or
[Ultralytics](https://github.com/ultralytics/ultralytics) packages. Following this,
you'll learn how to import these predictions into Supervision for image annotation
purposes.
Supervision provides a seamless process for annotating predictions generated by various
object detection and segmentation models. This guide shows how to perform inference
with the [Inference](https://github.com/roboflow/inference),
[Ultralytics](https://github.com/ultralytics/ultralytics) or
[Transformers](https://github.com/huggingface/transformers) packages. Following this,
you'll learn how to import these predictions into Supervision and use them to annotate
source image.
## Run Inference
![basic-annotation](https://media.roboflow.com/supervision_detect_and_annotate_example_1.png)
First, you'll need to obtain predictions from your object detection or segmentation model.
## Run Detection
First, you'll need to obtain predictions from your object detection or segmentation
model.
=== "Inference"
```python
import cv2
from inference import get_model
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
```
=== "Ultralytics"
@@ -23,123 +37,374 @@ First, you'll need to obtain predictions from your object detection or segmentat
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
image = cv2.imread(<PATH TO IMAGE>)
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
```
=== "Inference"
=== "Transformers"
```python
import cv2
from inference.models.utils import get_roboflow_model
import torch
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>)
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
```
## Load Predictions into Supervision
Now that we have predictions from a model, we can load them into Supervision.
=== "Ultralytics"
We can do so using the [`sv.Detections.from_ultralytics`](detection/core/#supervision.detection.core.Detections.from_ultralytics) method, which accepts model results from both detection and segmentation models.
```python
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
image = cv2.imread(<PATH TO IMAGE>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
```
=== "Inference"
We can do so using the [`sv.Detections.from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) method, which accepts model results from both detection and segmentation models.
```python
```{ .py hl_lines="2 8" }
import cv2
import supervision as sv
from inference.models.utils import get_roboflow_model
from inference import get_model
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
image = cv2.imread(<PATH TO IMAGE>)
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
```
You can conveniently load predictions from other computer vision frameworks and libraries using:
- [`from_deepsparse`](detection/core/#supervision.detection.core.Detections.from_deepsparse) ([Deepsparse](https://github.com/neuralmagic/deepsparse))
- [`from_detectron2`](detection/core/#supervision.detection.core.Detections.from_detectron2) ([Detectron2](https://github.com/facebookresearch/detectron2))
- [`from_mmdetection`](detection/core/#supervision.detection.core.Detections.from_mmdetection) ([MMDetection](https://github.com/open-mmlab/mmdetection))
- [`from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) ([Roboflow Inference](https://github.com/roboflow/inference))
- [`from_sam`](detection/core/#supervision.detection.core.Detections.from_sam) ([Segment Anything Model](https://github.com/facebookresearch/segment-anything))
- [`from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) ([HuggingFace Transformers](https://github.com/huggingface/transformers))
- [`from_yolo_nas`](detection/core/#supervision.detection.core.Detections.from_yolo_nas) ([YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md))
## Annotate Image
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](annotators/#supervision.annotators.core.LabelAnnotator) classes. If you are running the segmentation model [`sv.MaskAnnotator`](annotators/#supervision.annotators.core.MaskAnnotator) is a drop-in replacement for [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) that will allow you to draw masks instead of boxes.
=== "Ultralytics"
```python
We can do so using the [`sv.Detections.from_ultralytics`](detection/core/#supervision.detection.core.Detections.from_ultralytics) method, which accepts model results from both detection and segmentation models.
```{ .py hl_lines="2 8" }
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
image = cv2.imread(<PATH TO IMAGE>)
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
bounding_box_annotator = sv.BoundingBoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
model.model.names[class_id]
for class_id
in detections.class_id
]
annotated_image = bounding_box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
=== "Transformers"
We can do so using the [`sv.Detections.from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) method, which accepts model results from both detection and segmentation models.
```{ .py hl_lines="2 19-21" }
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(
transformers_results=results,
id2label=model.config.id2label)
```
You can load predictions from other computer vision frameworks and libraries using:
- [`from_deepsparse`](/latest/detection/core/#supervision.detection.core.Detections.from_deepsparse) ([Deepsparse](https://github.com/neuralmagic/deepsparse))
- [`from_detectron2`](/latest/detection/core/#supervision.detection.core.Detections.from_detectron2) ([Detectron2](https://github.com/facebookresearch/detectron2))
- [`from_mmdetection`](/latest/detection/core/#supervision.detection.core.Detections.from_mmdetection) ([MMDetection](https://github.com/open-mmlab/mmdetection))
- [`from_sam`](/latest/detection/core/#supervision.detection.core.Detections.from_sam) ([Segment Anything Model](https://github.com/facebookresearch/segment-anything))
- [`from_yolo_nas`](/latest/detection/core/#supervision.detection.core.Detections.from_yolo_nas) ([YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md))
## Annotate Image with Detections
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoxAnnotator`](/latest/annotators/#supervision.annotators.core.BoxAnnotator) and [`sv.LabelAnnotator`](/latest/annotators/#supervision.annotators.core.LabelAnnotator) classes.
=== "Inference"
```python
```{ .py hl_lines="10-16" }
import cv2
import supervision as sv
from inference.models.utils import get_roboflow_model
from inference import get_model
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
image = cv2.imread(<PATH TO IMAGE>)
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
bounding_box_annotator = sv.BoundingBoxAnnotator()
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = bounding_box_annotator.annotate(
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![Predictions plotted on an image](https://media.roboflow.com/supervision_annotate_example.png)
=== "Ultralytics"
## Display Annotated Image
```{ .py hl_lines="10-16" }
import cv2
import supervision as sv
from ultralytics import YOLO
To display the annotated image in Jupyter Notebook or Google Colab, use the [`sv.plot_image`](utils/notebook/#supervision.utils.notebook.plot_image) function.
model = YOLO("yolov8n.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
```python
sv.plot_image(annotated_image)
```
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Transformers"
```{ .py hl_lines="23-30" }
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(
transformers_results=results,
id2label=model.config.id2label)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![basic-annotation](https://media.roboflow.com/supervision_detect_and_annotate_example_1.png)
## Display Custom Labels
By default, [`sv.LabelAnnotator`](/latest/annotators/#supervision.annotators.core.LabelAnnotator)
will label each detection with its `class_name` (if possible) or `class_id`. You can
override this behavior by passing a list of custom `labels` to the `annotate` method.
=== "Inference"
```{ .py hl_lines="13-17 22" }
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
=== "Ultralytics"
```{ .py hl_lines="13-17 22" }
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
=== "Transformers"
```{ .py hl_lines="26-30 35" }
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(
transformers_results=results,
id2label=model.config.id2label)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
![custom-label-annotation](https://media.roboflow.com/supervision_detect_and_annotate_example_2.png)
## Annotate Image with Segmentations
If you are running the segmentation model
[`sv.MaskAnnotator`](/latest/annotators/#supervision.annotators.core.MaskAnnotator)
is a drop-in replacement for
[`sv.BoxAnnotator`](/latest/annotators/#supervision.annotators.core.BoxAnnotator)
that will allow you to draw masks instead of boxes.
=== "Inference"
```python
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-seg-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```python
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n-seg.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Transformers"
```python
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForSegmentation
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50-panoptic")
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_segmentation(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(
transformers_results=results,
id2label=model.config.id2label)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
![segmentation-annotation](https://media.roboflow.com/supervision_detect_and_annotate_example_3.png)
+325
View File
@@ -0,0 +1,325 @@
---
comments: true
---
# Detect Small Objects
This guide shows how to detect small objects
with the [Inference](https://github.com/roboflow/inference),
[Ultralytics](https://github.com/ultralytics/ultralytics) or
[Transformers](https://github.com/huggingface/transformers) packages using
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer).
<video controls>
<source src="https://media.roboflow.com/supervision_detect_small_objects_example.mp4" type="video/mp4">
</video>
## Baseline Detection
Small object detection in high-resolution images presents challenges due to the objects'
size relative to the image resolution.
=== "Inference"
```python
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8x-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```python
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8x.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Transformers"
```python
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForSegmentation
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<SOURCE_IMAGE_PATH>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image_slice.size
target_size = torch.tensor([[width, height]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
model.config.id2label[class_id]
for class_id
in detections.class_id
]
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
![basic-detection](https://media.roboflow.com/supervision_detect_small_objects_example_1.png)
## Input Resolution
Modifying the input resolution of images before detection can enhance small object
identification at the cost of processing speed and increased memory usage. This method
is less effective for ultra-high-resolution images (4K and above).
=== "Inference"
```{ .py hl_lines="5" }
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8x-1280")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```{ .py hl_lines="7" }
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8x.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
results = model(image, imgsz=1280)[0]
detections = sv.Detections.from_ultralytics(results)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![detection-with-high-input-resolution](https://media.roboflow.com/supervision_detect_small_objects_example_2.png)
## Inference Slicer
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer)
processes high-resolution images by dividing them into smaller segments, detecting
objects within each, and aggregating the results.
<video controls>
<source src="https://media.roboflow.com/supervision_detect_small_objects_example_2.mp4" type="video/mp4">
</video>
=== "Inference"
```{ .py hl_lines="9-14" }
import cv2
import numpy as np
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8x-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
results = model.infer(image_slice)[0]
return sv.Detections.from_inference(results)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```{ .py hl_lines="9-14" }
import cv2
import numpy as np
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8x.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
result = model(image_slice)[0]
return sv.Detections.from_ultralytics(result)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Transformers"
```{ .py hl_lines="13-28" }
import cv2
import torch
import numpy as np
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
image_slice = cv2.cvtColor(image_slice, cv2.COLOR_BGR2RGB)
image_slice = Image.fromarray(image_slice)
inputs = processor(images=image_slice, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image_slice.size
target_size = torch.tensor([[width, height]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
return sv.Detections.from_transformers(results)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
model.config.id2label[class_id]
for class_id
in detections.class_id
]
annotated_image = box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections, labels=labels)
```
![detection-with-inference-slicer](https://media.roboflow.com/supervision_detect_small_objects_example_3.png)
## Small Object Segmentation
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer) can perform segmentation tasks too.
=== "Inference"
```{ .py hl_lines="6 16 19-20" }
import cv2
import numpy as np
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8x-seg-640")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
results = model.infer(image_slice)[0]
return sv.Detections.from_inference(results)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```{ .py hl_lines="6 16 19-20" }
import cv2
import numpy as np
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8x-seg.pt")
image = cv2.imread(<SOURCE_IMAGE_PATH>)
def callback(image_slice: np.ndarray) -> sv.Detections:
result = model(image_slice)[0]
return sv.Detections.from_ultralytics(result)
slicer = sv.InferenceSlicer(callback = callback)
detections = slicer(image)
mask_annotator = sv.MaskAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = mask_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![detection-with-inference-slicer](https://media.roboflow.com/supervision-docs/inference-slicer-segmentation-example.png)
+455
View File
@@ -0,0 +1,455 @@
---
comments: true
status: new
---
With Supervision, you can load and manipulate classification, object detection, and
segmentation datasets. This tutorial will walk you through how to load, split, merge,
visualize, and augment datasets in Supervision.
## Download Dataset
In this tutorial, we will use a dataset from
[Roboflow Universe](https://universe.roboflow.com/), a public repository of
thousands of computer vision datasets. If you already have your dataset in
[COCO](https://roboflow.com/formats/coco-json),
[YOLO](https://roboflow.com/formats/yolov8-pytorch-txt),
or [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format, you can skip this
section.
```bash
pip install roboflow
```
Next, log into your Roboflow account and download the dataset of your choice in the
COCO, YOLO, or Pascal VOC format. You can customize the following code snippet with
your workspace ID, project ID, and version number.
=== "COCO"
```python
import roboflow
roboflow.login()
rf = roboflow.Roboflow()
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
dataset = project.version('<PROJECT_VERSION>').download("coco")
```
=== "YOLO"
```python
import roboflow
roboflow.login()
rf = roboflow.Roboflow()
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
dataset = project.version('<PROJECT_VERSION>').download("yolov8")
```
=== "Pascal VOC"
```python
import roboflow
roboflow.login()
rf = roboflow.Roboflow()
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
dataset = project.version('<PROJECT_VERSION>').download("voc")
```
## Load Dataset
The Supervision library provides convenient functions to load datasets in various
formats. If your dataset is already split into train, test, and valid subsets, you can
load each of those as separate [`sv.DetectionDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset)
instances.
=== "COCO"
We can do so using the [`sv.DetectionDataset.from_coco`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_coco) to load annotations in [COCO](https://roboflow.com/formats/coco-json) format.
```python
import supervision as sv
ds_train = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/train',
annotations_path=f'{dataset.location}/train/_annotations.coco.json',
)
ds_valid = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/valid',
annotations_path=f'{dataset.location}/valid/_annotations.coco.json',
)
ds_test = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/test',
annotations_path=f'{dataset.location}/test/_annotations.coco.json',
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
```
=== "YOLO"
We can do so using the [`sv.DetectionDataset.from_yolo`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_yolo) to load annotations in [YOLO](https://roboflow.com/formats/yolov8-pytorch-txt) format.
```python
import supervision as sv
ds_train = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/train/images',
annotations_directory_path=f'{dataset.location}/train/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_valid = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/valid/images',
annotations_directory_path=f'{dataset.location}/valid/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_test = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/test/images',
annotations_directory_path=f'{dataset.location}/test/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
```
=== "Pascal VOC"
We can do so using the [`sv.DetectionDataset.from_pascal_voc`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_pascal_voc) to load annotations in [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format.
```python
import supervision as sv
ds_train = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/train/images',
annotations_directory_path=f'{dataset.location}/train/labels'
)
ds_valid = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/valid/images',
annotations_directory_path=f'{dataset.location}/valid/labels'
)
ds_test = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/test/images',
annotations_directory_path=f'{dataset.location}/test/labels'
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
```
## Split Dataset
If your dataset is not already split into train, test, and valid subsets, you can
easily do so using the [`sv.DetectionDataset.split`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.split)
method. We can split it as follows, ensuring a random shuffle of the data.
```python
import supervision as sv
ds = sv.DetectionDataset(...)
len(ds)
# 1000
ds_train, ds = ds.split(split_ratio=0.8, shuffle=True)
ds_valid, ds_test = ds.split(split_ratio=0.5, shuffle=True)
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
```
## Merge Dataset
If you have multiple datasets that you would like to merge, you can do so using the
[`sv.DetectionDataset.merge`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.merge)
method.
=== "COCO"
```{ .py hl_lines="22-28" }
import supervision as sv
ds_train = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/train',
annotations_path=f'{dataset.location}/train/_annotations.coco.json',
)
ds_valid = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/valid',
annotations_path=f'{dataset.location}/valid/_annotations.coco.json',
)
ds_test = sv.DetectionDataset.from_coco(
images_directory_path=f'{dataset.location}/test',
annotations_path=f'{dataset.location}/test/_annotations.coco.json',
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
ds.classes
# ['person', 'bicycle', 'car', ...]
len(ds)
# 1000
```
=== "YOLO"
```{ .py hl_lines="25-31" }
import supervision as sv
ds_train = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/train/images',
annotations_directory_path=f'{dataset.location}/train/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_valid = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/valid/images',
annotations_directory_path=f'{dataset.location}/valid/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_test = sv.DetectionDataset.from_yolo(
images_directory_path=f'{dataset.location}/test/images',
annotations_directory_path=f'{dataset.location}/test/labels',
data_yaml_path=f'{dataset.location}/data.yaml'
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
ds.classes
# ['person', 'bicycle', 'car', ...]
len(ds)
# 1000
```
=== "Pascal VOC"
```{ .py hl_lines="22-28" }
import supervision as sv
ds_train = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/train/images',
annotations_directory_path=f'{dataset.location}/train/labels'
)
ds_valid = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/valid/images',
annotations_directory_path=f'{dataset.location}/valid/labels'
)
ds_test = sv.DetectionDataset.from_pascal_voc(
images_directory_path=f'{dataset.location}/test/images',
annotations_directory_path=f'{dataset.location}/test/labels'
)
ds_train.classes
# ['person', 'bicycle', 'car', ...]
len(ds_train), len(ds_valid), len(ds_test)
# 800, 100, 100
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
ds.classes
# ['person', 'bicycle', 'car', ...]
len(ds)
# 1000
```
## Iterate over Dataset
There are two ways to loop over a `sv.DetectionDataset`: using a direct
[for loop](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.__iter__)
called on the `sv.DetectionDataset` instance or loading `sv.DetectionDataset` entries
[by index](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.__getitem__).
```python
import supervision as sv
ds = sv.DetectionDataset(...)
# Option 1
for image_path, image, annotations in ds:
... # Process each image and its annotations
# Option 2
for idx in range(len(ds)):
image_path, image, annotations = ds[idx]
... # Process the image and annotations at index `idx`
```
## Visualize Dataset
The Supervision library provides tools for easily visualizing your detection dataset.
You can create a grid of annotated images to quickly inspect your data and labels.
First, initialize the [`sv.BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator)
and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator).
Then, iterate through a subset of the dataset (e.g., the first 25 images), drawing
bounding boxes and class labels on each image. Finally, combine the annotated images
into a grid for display.
```python
import supervision as sv
ds = sv.DetectionDataset(...)
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_images = []
for i in range(16):
_, image, annotations = ds[i]
labels = [ds.classes[class_id] for class_id in annotations.class_id]
annotated_image = image.copy()
annotated_image = box_annotator.annotate(annotated_image, annotations)
annotated_image = label_annotator.annotate(annotated_image, annotations, labels)
annotated_images.append(annotated_image)
grid = sv.create_tiles(
annotated_images,
grid_size=(4, 4),
single_tile_size=(400, 400),
tile_padding_color=sv.Color.WHITE,
tile_margin_color=sv.Color.WHITE
)
```
![visualize-dataset](https://media.roboflow.com/supervision-docs/visualize-dataset.png)
## Save Dataset
=== "COCO"
We can do so using the [`sv.DetectionDataset.as_coco`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_coco) method to save annotations in [COCO](https://roboflow.com/formats/coco-json) format.
```python
import supervision as sv
ds = sv.DetectionDataset(...)
ds.as_coco(
images_directory_path='<IMAGE_DIRECTORY_PATH>',
annotations_path='<ANNOTATIONS_PATH>'
)
```
=== "YOLO"
We can do so using the [`sv.DetectionDataset.as_yolo`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_yolo) method to save annotations in [YOLO](https://roboflow.com/formats/yolov8-pytorch-txt) format.
```python
import supervision as sv
ds = sv.DetectionDataset(...)
ds.as_yolo(
images_directory_path='<IMAGE_DIRECTORY_PATH>',
annotations_directory_path='<ANNOTATIONS_DIRECTORY_PATH>',
data_yaml_path='<DATA_YAML_PATH>'
)
```
=== "Pascal VOC"
We can do so using the [`sv.DetectionDataset.as_pascal_voc`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_pascal_voc) method to save annotations in [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format.
```python
import supervision as sv
ds = sv.DetectionDataset(...)
ds.as_pascal_voc(
images_directory_path='<IMAGE_DIRECTORY_PATH>',
annotations_directory_path='<ANNOTATIONS_DIRECTORY_PATH>'
)
```
## Augment Dataset
In this section, we'll explore using Supervision in combination with Albumentations to
augment our dataset. Data augmentation is a common technique in computer vision to
increase the size and diversity of training datasets, leading to improved model
performance and generalization.
```bash
pip install augmentation
```
Albumentations provides a flexible and powerful API for image augmentation. The core of
the library is the [`Compose`](https://albumentations.ai/docs/api_reference/full_reference/?h=compose#albumentations.core.composition.Compose)
class, which allows you to chain multiple image transformations together. Each
transformation is defined using a dedicated class, such as
[`HorizontalFlip`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.geometric.transforms.HorizontalFlip),
[`RandomBrightnessContrast`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.transforms.RandomBrightnessContrast),
or [`Perspective`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.geometric.transforms.Perspective).
```python
import albumentations as A
augmentation = A.Compose(
transforms=[
A.Perspective(p=0.1),
A.HorizontalFlip(p=0.5),
A.RandomBrightnessContrast(p=0.5)
],
bbox_params=A.BboxParams(
format='pascal_voc',
label_fields=['category']
),
)
```
The key is to set `format='pascal_voc'`, which corresponds to the
`[x_min, y_min, x_max, y_max]` bounding box format used in Supervision.
```python
import numpy as np
import supervision as sv
from dataclasses import replace
ds = sv.DetectionDataset(...)
_, original_image, original_annotations = ds[0]
output = augmentation(
image=original_image,
bboxes=original_annotations.xyxy,
category=original_annotations.class_id
)
augmented_image = output['image']
augmented_annotations = replace(
original_annotations,
xyxy=np.array(output['bboxes']),
class_id=np.array(output['category'])
)
```
![augment-dataset](https://media.roboflow.com/supervision-docs/augment-dataset.png)
+298
View File
@@ -0,0 +1,298 @@
---
comments: true
---
# Save Detections
Supervision enables an easy way to save detections in .CSV and .JSON files for offline
processing. This guide demonstrates how to perform video inference using the
[Inference](https://github.com/roboflow/inference),
[Ultralytics](https://github.com/ultralytics/ultralytics) or
[Transformers](https://github.com/huggingface/transformers) packages and save their results with
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) and
[`sv.JSONSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink).
## Run Detection
First, you'll need to obtain predictions from your object detection or segmentation
model. You can learn more on this topic in our
[How to Detect and Annotate](/latest/how_to/detect_and_annotate.md) guide.
=== "Inference"
```python
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
for frame in frames_generator:
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
```
=== "Ultralytics"
```python
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
for frame in frames_generator:
results = model(frame)[0]
detections = sv.Detections.from_ultralytics(results)
```
=== "Transformers"
```python
import torch
import supervision as sv
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
for frame in frames_generator:
frame = sv.cv2_to_pillow(frame)
inputs = processor(images=frame, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = frame.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
```
## Save Detections as CSV
To save detections to a `.CSV` file, open our
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
and then pass the
[`sv.Detections`](/latest/detection/core/#supervision.detection.core.Detections)
object resulting from the inference to it. Its fields are parsed and saved on disk.
=== "Inference"
```{ .py hl_lines="7 12" }
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame in frames_generator:
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
sink.append(detections, {})
```
=== "Ultralytics"
```{ .py hl_lines="7 12" }
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame in frames_generator:
results = model(frame)[0]
detections = sv.Detections.from_ultralytics(results)
sink.append(detections, {})
```
=== "Transformers"
```{ .py hl_lines="9 23" }
import torch
import supervision as sv
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame in frames_generator:
frame = sv.cv2_to_pillow(frame)
inputs = processor(images=frame, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = frame.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
sink.append(detections, {})
```
| x_min | y_min | x_max | y_max | class_id | confidence | tracker_id | class_name |
|---------|----------|---------|----------|----------|------------|------------|------------|
| 2941.14 | 1269.31 | 3220.77 | 1500.67 | 2 | 0.8517 | | car |
| 944.889 | 899.641 | 1235.42 | 1308.80 | 7 | 0.6752 | | truck |
| 1439.78 | 1077.79 | 1621.27 | 1231.40 | 2 | 0.6450 | | car |
## Custom Fields
Besides regular fields in
[`sv.Detections`](/latest/detection/core/#supervision.detection.core.Detections),
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
also allows you to add custom information to each row, which can be passed via the
`custom_data` dictionary. Let's utilize this feature to save information about the
frame index from which the detections originate.
=== "Inference"
```{ .py hl_lines="8 12" }
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
sink.append(detections, {"frame_index": frame_index})
```
=== "Ultralytics"
```{ .py hl_lines="8 12" }
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
results = model(frame)[0]
detections = sv.Detections.from_ultralytics(results)
sink.append(detections, {"frame_index": frame_index})
```
=== "Transformers"
```{ .py hl_lines="10 23" }
import torch
import supervision as sv
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
frame = sv.cv2_to_pillow(frame)
inputs = processor(images=frame, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = frame.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
sink.append(detections, {"frame_index": frame_index})
```
| x_min | y_min | x_max | y_max | class_id | confidence | tracker_id | class_name | frame_index |
|---------|----------|---------|----------|----------|------------|------------|------------|-------------|
| 2941.14 | 1269.31 | 3220.77 | 1500.67 | 2 | 0.8517 | | car | 0 |
| 944.889 | 899.641 | 1235.42 | 1308.80 | 7 | 0.6752 | | truck | 0 |
| 1439.78 | 1077.79 | 1621.27 | 1231.40 | 2 | 0.6450 | | car | 0 |
## Save Detections as JSON
If you prefer to save the result in a `.JSON` file instead of a `.CSV` file, all you
need to do is replace
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
with
[`sv.JSONSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink).
=== "Inference"
```{ .py hl_lines="7" }
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
sink.append(detections, {"frame_index": frame_index})
```
=== "Ultralytics"
```{ .py hl_lines="7" }
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
results = model(frame)[0]
detections = sv.Detections.from_ultralytics(results)
sink.append(detections, {"frame_index": frame_index})
```
=== "Transformers"
```{ .py hl_lines="9" }
import torch
import supervision as sv
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
for frame_index, frame in enumerate(frames_generator):
frame = sv.cv2_to_pillow(frame)
inputs = processor(images=frame, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = frame.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
sink.append(detections, {"frame_index": frame_index})
```
+47 -12
View File
@@ -1,19 +1,20 @@
---
template: index.html
comments: true
hide:
- navigation
- toc
---
<div align="center">
<p>
<a align="center" href="" target="_blank">
<img
width="850"
src="https://media.roboflow.com/open-source/supervision/rf-supervision-banner.png?updatedAt=1678995927529"
>
</a>
</p>
<div class="md-typeset">
<h1></h1>
</div>
<div align="center" id="logo">
<a align="center" href="" target="_blank">
<img width="850"
src="https://media.roboflow.com/open-source/supervision/rf-supervision-banner.png?updatedAt=1678995927529">
</a>
</div>
## 👋 Hello
@@ -29,7 +30,7 @@ We write your reusable computer vision tools. Whether you need to load your data
## 💻 Install
You can install `supervision` with pip in a
You can install `supervision` in a
[**Python>=3.8**](https://www.python.org/) environment.
!!! example "pip install (recommended)"
@@ -45,7 +46,25 @@ You can install `supervision` with pip in a
If you require the full version of `supervision` with GUI support you can install the desktop version. This version includes the GUI components of OpenCV, allowing you to display images and videos on the screen.
```bash
pip install supervision[desktop]
pip install "supervision[desktop]"
```
!!! example "conda/mamba install"
=== "conda"
[![conda-recipe](https://img.shields.io/badge/recipe-supervision-green.svg)](https://anaconda.org/conda-forge/supervision) [![conda-downloads](https://img.shields.io/conda/dn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision) [![conda-version](https://img.shields.io/conda/vn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision) [![conda-platforms](https://img.shields.io/conda/pn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision)
```bash
conda install -c conda-forge supervision
```
=== "mamba"
[![mamba-recipe](https://img.shields.io/badge/recipe-supervision-green.svg)](https://anaconda.org/conda-forge/supervision) [![mamba-downloads](https://img.shields.io/conda/dn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision) [![mamba-version](https://img.shields.io/conda/vn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision) [![mamba-platforms](https://img.shields.io/conda/pn/conda-forge/supervision.svg)](https://anaconda.org/conda-forge/supervision)
```bash
mamba install -c conda-forge supervision
```
!!! example "git clone (for development)"
@@ -87,7 +106,6 @@ You can install `supervision` with pip in a
poetry install --extras "desktop"
```
## 🚀 Quickstart
<div class="grid cards" markdown>
@@ -108,6 +126,14 @@ You can install `supervision` with pip in a
[:octicons-arrow-right-24: Tutorial](how_to/track_objects.md)
- __Detect Small Objects__
---
Learn how to detect small objects in images
[:octicons-arrow-right-24: Tutorial](how_to/detect_small_objects.md)
- > __Count Objects Crossing Line__
---
@@ -120,4 +146,13 @@ You can install `supervision` with pip in a
Master the techniques to selectively filter and focus on objects within a specific zone
- **Cheatsheet**
***
Access a quick reference guide to the most common `supervision` functions
[:octicons-arrow-right-24: Cheatsheet](https://roboflow.github.io/cheatsheet-supervision/)
</div>
+1 -1
View File
@@ -55,7 +55,7 @@ document.addEventListener("DOMContentLoaded", function () {
let authorAvatarsHTML = authorDataArray.map((authorData, index) => {
const marginLeft = index === 0 ? '0' : '-10px';
const zIndex = 100 - index;
const zIndex = 4 - index;
return `
<div
class="author-container"
+96
View File
@@ -0,0 +1,96 @@
---
comments: true
---
# Annotators
=== "VertexAnnotator"
```python
import supervision as sv
image = ...
key_points = sv.KeyPoints(...)
vertex_annotator = sv.VertexAnnotator(
color=sv.Color.GREEN,
radius=10
)
annotated_frame = vertex_annotator.annotate(
scene=image.copy(),
key_points=key_points
)
```
<div class="result" markdown>
![vertex-annotator-example](https://media.roboflow.com/supervision-annotator-examples/vertex-annotator-example.png){ align=center width="800" }
</div>
=== "EdgeAnnotator"
```python
import supervision as sv
image = ...
key_points = sv.KeyPoints(...)
edge_annotator = sv.EdgeAnnotator(
color=sv.Color.GREEN,
thickness=5
)
annotated_frame = edge_annotator.annotate(
scene=image.copy(),
key_points=key_points
)
```
<div class="result" markdown>
![edge-annotator-example](https://media.roboflow.com/supervision-annotator-examples/edge-annotator-example.png){ align=center width="800" }
</div>
=== "VertexLabelAnnotator"
```python
import supervision as sv
image = ...
key_points = sv.KeyPoints(...)
vertex_label_annotator = sv.VertexLabelAnnotator(
color=sv.Color.GREEN,
text_color=sv.Color.BLACK,
border_radius=5
)
annotated_frame = vertex_label_annotator.annotate(
scene=image.copy(),
key_points=key_points
)
```
<div class="result" markdown>
![vertex-label-annotator-example](https://media.roboflow.com/supervision-annotator-examples/vertex-label-annotator-example.png){ align=center width="800" }
</div>
<div class="md-typeset">
<h2><a href="#supervision.keypoint.annotators.VertexAnnotator">VertexAnnotator</a></h2>
</div>
:::supervision.keypoint.annotators.VertexAnnotator
<div class="md-typeset">
<h2><a href="#supervision.keypoint.annotators.EdgeAnnotator">EdgeAnnotator</a></h2>
</div>
:::supervision.keypoint.annotators.EdgeAnnotator
<div class="md-typeset">
<h2><a href="#supervision.keypoint.annotators.VertexLabelAnnotator">VertexLabelAnnotator</a></h2>
</div>
:::supervision.keypoint.annotators.VertexLabelAnnotator
+8
View File
@@ -0,0 +1,8 @@
---
comments: true
status: new
---
# Keypoint Detection
:::supervision.keypoint.core.KeyPoints
-22
View File
@@ -1,22 +0,0 @@
---
comments: true
---
#  Detection Metrics
!!! warning
Evaluation API is still fluid and may change. If you use Evaluation API in your project until further notice, freeze the
`supervision` version in your `requirements.txt` or `setup.py`.
<div class="md-typeset">
<h2><a href="#supervision.metrics.detection.ConfusionMatrix">ConfusionMatrix</a></h2>
</div>
:::supervision.metrics.detection.ConfusionMatrix
<div class="md-typeset">
<h2><a href="#supervision.annotators.core.MeanAveragePrecision">MeanAveragePrecision</a></h2>
</div>
:::supervision.metrics.detection.MeanAveragePrecision
+277 -82
View File
@@ -2,7 +2,9 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "TfUZYrKbqlF7"
},
"source": [
"# Annotate Video with Detections\n",
"\n",
@@ -10,9 +12,72 @@
"\n",
"[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/annotate-video-with-detections.ipynb)\n",
"\n",
"One of the most common requirements of computer vision applications is detecting objects in images and displaying bounding boxes around those objects. In this cookbook we'll walk through the steps on how to utilize the open source Roboflow ecosystem to accomplish this task on a video. Let's dive in! \n",
"One of the most common requirements of computer vision applications is detecting objects in images and displaying bounding boxes around those objects. In this cookbook we'll walk through the steps on how to utilize the open source Roboflow ecosystem to accomplish this task on a video. Let's dive in!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rUo1u17yqlF9"
},
"source": [
"## Before you start\n",
"\n",
"## Installing Dependencies \n",
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0Er370WTqlF9",
"outputId": "57b00a1e-87ea-4912-b4f9-5b94f0e6bf48",
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fri Feb 23 03:15:00 2024 \n",
"+---------------------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
"|-----------------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|=========================================+======================+======================|\n",
"| 0 Tesla V100-SXM2-16GB Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 33C P0 24W / 300W | 0MiB / 16384MiB | 0% Default |\n",
"| | | N/A |\n",
"+-----------------------------------------+----------------------+----------------------+\n",
" \n",
"+---------------------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=======================================================================================|\n",
"| No running processes found |\n",
"+---------------------------------------------------------------------------------------+\n"
]
}
],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WuSnN7ejqlF-"
},
"source": [
"## Installing Dependencies\n",
"\n",
"In this cookbook we'll be utilizing the open source packages [Inference](https://inference.roboflow.com/) and [Supervision](https://supervision.roboflow.com/latest/) to accomplish our goals. Let's get those installed in our notebook with pip."
]
@@ -21,40 +86,68 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1YgQGum4qlF-",
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"\n",
"!pip install -q inference \"supervision[assets]\""
"!pip install -q inference-gpu \"supervision[assets]\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "PZRnzh7EqlF-"
},
"source": [
"## Download a Video Asset\n",
"\n",
"First, let's download a video that we can detect objects in. Supervision comes with a great utility called Assets to help us hit the ground running. Wehn we run this script, the video is saved in our local directory and can be accessed with the variable `path_to_video`."
"First, let's download a video that we can detect objects in. Supervision comes with a great utility called Assets to help us hit the ground running. When we run this script, the video is saved in our local directory and can be accessed with the variable `path_to_video`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"metadata": {
"id": "NMg4KOFiqlF-"
},
"outputs": [],
"source": [
"from supervision.assets import download_assets, VideoAssets\n",
"\n",
"# Download a supervision video asset \n",
"# Download a supervision video asset\n",
"path_to_video = download_assets(VideoAssets.PEOPLE_WALKING)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "x93oxF0zrQOU"
},
"source": [
"As a result, we've downloaded a video. Let's take a look at the video below. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
"\n",
"<div style=\"display: flex; justify-content: center;\">\n",
" <video controls width=\"1000\"\">\n",
" <source\n",
" src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\"\n",
" type=\"video/mp4\"\n",
" >\n",
" </video>\n",
"</div>\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T8bU7fQ6qlF-"
},
"source": [
"## Detecting Objects\n",
"\n",
@@ -63,128 +156,230 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "hlONWHfQqlF-",
"outputId": "81b277fb-ea7c-4242-f49d-5af0c665ea58"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detections(xyxy=array([[1140., 951., 1245., 1079.],\n",
" [ 666., 648., 745., 854.],\n",
" [ 34., 794., 142., 990.],\n",
" [1140., 505., 1211., 657.],\n",
" [ 260., 438., 332., 612.],\n",
" [1413., 702., 1523., 887.],\n",
" [1462., 472., 1543., 643.],\n",
" [1446., 318., 1516., 483.],\n",
" [ 753., 451., 821., 623.],\n",
" [ 924., 172., 983., 307.],\n",
" [1791., 144., 1852., 275.],\n",
" [ 93., 132., 146., 251.],\n",
" [ 708., 240., 765., 388.],\n",
" [ 200., 44., 267., 161.],\n",
" [1204., 131., 1255., 266.],\n",
" [ 569., 267., 628., 408.],\n",
" [1163., 150., 1210., 280.],\n",
" [ 799., 78., 847., 204.],\n",
" [1690., 152., 1751., 283.],\n",
" [ 344., 495., 396., 641.],\n",
" [1722., 77., 1782., 178.]]),\n",
" mask=None,\n",
" confidence=array([0.83215541, 0.80572134, 0.7919845 , 0.7912274 , 0.77121079,\n",
" 0.7599591 , 0.75711554, 0.75494027, 0.73076195, 0.71452248,\n",
" 0.69572842, 0.65269446, 0.63952065, 0.62914598, 0.61361706,\n",
" 0.5968492 , 0.55311316, 0.5470854 , 0.54070991, 0.52209878,\n",
" 0.41217673]),\n",
" class_id=array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),\n",
" tracker_id=None,\n",
" data={'class_name': array(['person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person'], dtype='<U6')})\n"
]
}
],
"source": [
"import supervision as sv\n",
"from supervision.assets import download_assets, VideoAssets\n",
"from inference.models.utils import get_roboflow_model\n",
"\n",
"\n",
"if __name__ == \"__main__\":\n",
" # Download the video asset from Supervision assets.\n",
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
"# Load a yolov8 model from roboflow.\n",
"model = get_roboflow_model(\"yolov8s-640\")\n",
"\n",
" # Load a yolov8 nano model from roboflow.\n",
" model = get_roboflow_model(\"yolov8n-640\")\n",
"# Create a frame generator and video info object from supervision utilities.\n",
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
"\n",
" # Create a frame generator and video info object from supervision utilities.\n",
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
"# Yield a single frame from the generator.\n",
"frame = next(frame_generator)\n",
"\n",
" # Yield a single frame from the generator.\n",
" frame = next(frame_generator)\n",
"# Run inference on our frame\n",
"result = model.infer(frame)[0]\n",
"\n",
" # Run inference on our frame.\n",
" result = model.infer(frame)[0]\n",
"# Parse result into detections data model.\n",
"detections = sv.Detections.from_inference(result)\n",
"\n",
" # Parse result into detections data model.\n",
" detections = sv.Detections.from_inference(result)\n",
"\n",
" # Display the detections data model.\n",
" print(detections)"
"# Pretty Print the resulting detections.\n",
"from pprint import pprint\n",
"pprint(detections)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "mL7QrQvyqlF-"
},
"source": [
"First, we load our model using the method `get_roboflow_model()`. Notice how we pass in a `model_id`? We're using an [alias](https://inference.roboflow.com/reference_pages/model_aliases/) here. This is where we can pass in other models from Roboflow Universe like this [rock, paper, scissors](https://universe.roboflow.com/roboflow-58fyf/rock-paper-scissors-sxsw) model utilizing our roboflow api key. \n",
"First, we load our model using the method `get_roboflow_model()`. Notice how we pass in a `model_id`? We're using an [alias](https://inference.roboflow.com/reference_pages/model_aliases/) here. This is where we can pass in other models from Roboflow Universe like this [rock, paper, scissors](https://universe.roboflow.com/roboflow-58fyf/rock-paper-scissors-sxsw) model utilizing our roboflow api key.\n",
"\n",
"```\n",
"model = get_roboflow_mode(\n",
" model_id=\"rock-paper-scissors-sxsw/11\", \n",
" model_id=\"rock-paper-scissors-sxsw/11\",\n",
" api_key=\"roboflow_private_api_key\"\n",
")\n",
"```\n",
"\n",
"If you don't have an api key, you can [create an free Roboflow account](https://app.roboflow.com/login). This model wouldn't be much help with detecting people, but it's a nice exercise to see how our code becomes model agnostic!\n",
"\n",
"We then create a `frame_generator` object and yeild a single frame for inference using `next()`. We pass our frame to `model.infer()` to run inference, then pass that data into a little helpfer function called `sv.Detections.from_inference()` to parse it. Lastly we print our detections to show we are in fact detecting a few people in the frame! \n",
"We then create a `frame_generator` object and yeild a single frame for inference using `next()`. We pass our frame to `model.infer()` to run inference. After, we pass that data into a little helpfer function called `sv.Detections.from_inference()` to parse it. Lastly we print our detections to show we are in fact detecting a few people in the frame!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tM6geyUBwdej"
},
"source": [
"## Annotaing the Frame with Bounding Boxes\n",
"\n",
"## Saving Bounding Boxes to the Video\n",
"Now that we're detecting images, let's get to the fun part. Let's annotate the frame and display the bounding boxes on the frame."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "8gkUxKei0RaV",
"outputId": "84a95332-70cf-4f26-c4bb-15479afafd04"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9WYxlyXUeCn8Re58hz8mTY2VlZdY8Vw/V7InN7ubUFEnJFEVZlGzYhmRd0LgPF1cGBBgG/OIXPxr478sv3xf5vxANSpYlSpRJUCRFimK3mt3NLrK75p5q6qrKzMrMyso5z7x3/A+xV+wVsWOfc6rlHz8E5CpknXP2EMOKFWuKFSuEUkphF3ZhF3ZhF3ZhF3ZhF3ZhF3ZhF3bhHynI/383YBd2YRd2YRd2YRd2YRd2YRd2YRd24R8Cu4btLuzCLuzCLuzCLuzCLuzCLuzCLvyjhl3Ddhd2YRd2YRd2YRd2YRd2YRd2YRf+UcOuYbsLu7ALu7ALu7ALu7ALu7ALu7AL/6hh17DdhV3YhV3YhV3YhV3YhV3YhV3YhX/UsGvY7sIu7MIu7MIu7MIu7MIu7MIu7MI/atg1bHdhF3ZhF3ZhF3ZhF3ZhF3ZhF3bhHzXsGra7sAu7sAu7sAu7sAu7sAu7sAu78I8adg3bXdiFXdiFXdiFXdiFXdiFXdiFXfhHDeGgD/6n//Sfet4XQlifeaCUst6hPyklCoUCpJSZ+73KiaII3W4XURRBKWWVyd93y6I66VoYhgjDMPOMr14hBJRS5g8ApJT40z/9U1y+fBnPPPMMDh48iImJCQwNDZl6+LtbW1vodrvY2trC/Pw8Zmdn8eSTT6JQKJjnpJQWPqh+Xm8cx5m+PMw48LL4O3TdxSO/F0UR4ji23o/jGJ1OB1EUmTGJ4xgXL17Ej370I5TLZUxNTeGzn/0syuUy7t69i4sXL2J5eRlBEEBKacorFov45V/+ZTz55JPodrtWW3i9ABAEAcIwxH//7/8dH3zwAWZnZzEyMoJGo4EwDFEqldDtdtHpdNBsNtFsNrG+vo4gCDA2Nobf//3fx/j4OOI4tujDxb+LQ3q+UCggCAKrTf3GgnAIpPRXKBS878RxjG63a+rkdE/9D4LAqpPPLSklgiAw9MefIdy6NORrP6cbfr9YLJr+0/O+Z93+cxzQ/Cec+p53P5VS6HQ6iOMYYdibnfVqjxACpVLJ7isUFPxt4UBj4/IPX93uNd7+UqmU+0yvthMdCCEwNDSEhYUFfOMb3zDlEm0CwPDwMH79138dExMT3vbx8pVSKBaL+Iu/+At873vfQ6FQwBe/+EW88MILGBsbw3/5L/8F169fx4kTJ/Bv/+2/RalUMvyoF+92geZzEASG/vPAbR//Tn313c+DOI6xsLCAjY0NrKys4O7du5ienka1WkWhUPDOwziOsWfPHhw7dsyqK4oiFItFi2fwscvrOwHNA6IDPiac7xHQvObPtVotVCqVjDzm8z6PFgGg1WoZWTzo+HEgvkRz0e2/W6YPv51OB8ViMVf+uuDKe5I7hUIBgQwhkD8n3bbR7263a2SK+yyfky6v4rI+iiKUy+Xcvva73ul0oJQyfJuXn4cLt7x2u214I+FHQQFQcNHCy+V4AGDed2UHv8bbxK+1221ThpQSgQgzzxDEKs7w3G63a8ajWCzm0gCNx/nz5/GHf/iHiOMYMzMz+OpXv4rf/d3/DSPDI3jvg3fxjW98A/v370etVkOpVMKtW7dw+vRpjIyMII5jM3/depRSRlYSn+lF23lAuhHHo1uP+9uHY3cMSB9z77k8wFc+b3/evCOdSyll+I5b5yB4cOuncrm+1U/n8PWB65uuvpM3Tj7c9ptX/fDEy3Hr9fXPB9Qfmn957eZ2AOdFURQZ/sHvtdttvPzyy7h8+bKRJ/SXJxeiKMKpU6fwL/7FvzDj7psjUsqMDuTigNfVy/YQQiAMQzOG9Pd//h+/1xNvBAMbtv2AGsMVWwLOMN17xKiJqLmgzjPUOFKoTFKIfROa3vcRFiGw14DkMR4+YKVSCZ/+9Kdx+fJlvPnmmzh//jzCMMTw8DCGhoZQKBTQbDbR7XZRr9fRaDRw8OBBPP300xgdHTVM32VE/YDjz8V5LwPDFUg+XOX1n9/nY8qFLzdYhBA4ceIEWq0WhoeHMTY2BiklFhYW8Oabb2JnZwelUgkTExOQUmJ5ebkns/Hhp1wu48GDB6jX60YpBbTB1+l0LEZHYw4AY2NjiOMYS0tLmJyctOrgjLaXMOUKUB7kGYzdbtcyPAl3ecKG7pFwJIOAzz3fHOhnHH1U4HX2U2J979IfGV55Qpd/50KWjHyXbzyswtHPoOrVB3I4fNT3uaDo9yz/9JURxzGKxSKKxSLCMLSEGin9d+7cMXOwV/mE23a7bfo5NzcHpbTBe+TIEdy4cQPvv/8+fv7zn+Oll14yz1J7+iltvB56Ps+xwfvKf9M7vZS2XuVxBSIMQ7RaLeNkcXky4ZHX/TD1DUKL/YzQfvW5fGDQ9+jZKIosB+sgtM1xQnKY18MNhkHKIb426PO8LvqulMo1anuVR7yfOw19/MSHH/6dHKlc7vQDl/fTXCAe2c+o5fd4Pyy9AgJK9Ha20LOFQiEzP108uM5Itw9BEBijAwqJYZ3l03l9Iactd+C5xhrJUdIptre3USgU0Ol0cOHCBYyNjeGlz76EN15/Az/5yU/QaDQQxwrPPPM0vvSlL2FiYsKaz3m078pnjod+Y0P3iZ7y5q77O+85Ps+pfJ9+2w98zw5CZ7zOPH1lEODzK6++QdvBDW2uu/l49f8q4H3nNOPSKm9rP6D+BEHQUx5S/a5uRLyLl0WO3xdffBG3b9/G+vp6xsDMg+npaZTLZa+u00ve+H67OgmVwcv7h8L/MsN2EPAxCkK6j1nxZ/h31+p3jRAXuNHslsW9AkC+stKPCUdRhNOnT+OFF17Aj370I3OtXq8jCALMzMxgbm7OEvCdTgdHjhxBs9nEyMjIQynXLjPjDNdHYIMwTN5Hl4HnTV6fQUxOAlrJAvTq62OPPYbt7W0j8IeHh3Ho0CGEYYiRkRHUajWsrKzgwYMHFlOgseZ1kbDk9V6/fh3r6+tWe4rFIkqlEprNpqW0kPI/OTmJTqeD69ev47HHHrM8Upxe8hQTLkzcNvpw5BuDUqlkjFq3nDyh5jIv3h5eNp8TpJy4Zbnv+crqRf+u8e/iIW8+0Xe+yuG2Ie8a4cAVYr3ANf55eXlj6OuDK0QGqT8Pf/T+IEo81ZmnVJFzpVqtGidSp9NBq9Uyz0RRhNu3b+ORRx5BuVzOlOUatVtbW1hZWTFKA60AAdqZRDT4wQcf4Pnnnzdt7DdvfHggA7PXir1LY1yJALJ49PE4Dt1uF+12G81mE1JKjIyMmBUmH/Sbl/2UqEEULNcwcD/z+Ard47jpZfj56gVsWh5UyeDziq/a9jL88ngEGUFcLxiEl/raEYiUlgZRurmxRrIrL5KFO+N4v0iGkGPJdTDxunxGEf/rdDoAtPzM63MeEA59vKUXn+M4JzlLegw9Q99JppDi7NMzLAPD0/csTWR1Co4bahsvNwgC3Lp1C//X/+v/wuLiIhT0/a2tLczNzeFv//ZvcenSJQwNDeGZZ57BK6+8gvX1dbz22ms4ffo0Dhw4YJxyvP+uDCQjzOeoGXSuUDnuamc/485Xlys/fPqYr/68+e++4849nz7otqnXPOulX/MyfHW69bj1uZGRnB/yej6q4dSLn/qAG+w+Xj1IOdQn4xjKaRfRJOcZ7rwiXVoIgcnJSezs7KDRaHjbwNsnhcTk5KRXLvvqcO/za7SAR9Fk7hgNwqMHgf/lhm0/YnYJjyMjz6vrUyTd+vKYJDdmXKZNK7VuPb4+EVHkAXlmP/nJT+L8+fPY3t5OQ2+CAL/0S7+ECxcu4MKFC4apNZtNDA8PY8+ePbl19jI4AOQKLp+g5OW4xORTMvIMGy5gXKOErgEwXv8oiqyVEP7eM888Y97vdDrY2NiwjEtqG1/RJiBjsFwu45133sGPf/xjbGxsoFarmTaSUkFhkqQAK6UNSiklKpUK3n//fWxsbGBiYsJ4f/PwwfGYp5z1+k3vdjodlEolK0zM956Ld94Pd1XJHSsexcCjGvi4+kKo8wweX995tEMvhdkHnLnxPvZi4vy76/njbfaBb7w+qqCjdtMKl+9ev2t588gH1F+XDoUQxnFTLpdRqVQwMjKCubk5TE5OGkFC4Z1ra2u4e/cuTp8+7VVGwzDExsYGzp07h1defgWra6uYnp7G6uoqJiYmMDY2hp2dHdy9e9e8Ozc3h9XVVVQqlYwj0RdNwtvvU2zcfvdSUDheBlE++H1yvrXbbXQ6HdRqNayvr6NcLmd4MH+/2Wwavsav96L/PCWT94nmdh5fcfHkwyXHlU95cA0Et6xeSlQecNyTfHOVy0FDFguFgjEy6LmHUXTo2W63i1CGAPJXr/OukczJ0y98NMGNLfp08cBhELonpZbKdGWtO3/c94FUwabVHzIvB+HxXCfjIc337t3DxYsXsbi4iOnpaXz2s5/F0NBQhj9x/MVxDBXHgBOdY9Ew0lV2V2fhK50cp7T16Y/+6I9wb/GeKateryMMQxw5cgRRFKHRaKDVauHtt9/G5uYmgiBAo9HAD37wAzzzzDNGRxgUBo1C8EEvR8Agv33jxXHda471kzH8vUF1GreMXs6xXvUTnQyKW197eZSBT5fKm7t5smJQXujaNT5dhvPdPDnl3qff5HB1tyNwOvLhjp4l5/T29jampqawvr6Ozc1NhGHYs/5iqYjx8XEv33HrGAR/JCNcXsbvu3rtw8qAgQ3bhym037t5IV4+Jk2KY6+9A/Qcv8+/5xlEeYTlKk6+Z1xQSu9vmp2dxdNPP42XX37ZGEiNRgPtdhtf+cpXTAiAlBLb29vY3NzE8PCw8Q73U35cPOat9LhCgb73YLine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create a bounding box annotator object.\n",
"bounding_box = sv.BoundingBoxAnnotator()\n",
"\n",
"Now comes the fun part. Let's wrap up our code by utilizing `Annotators` and a `VideoSink` to draw bounding boxes and save the resulting video respectively. Take a peak at the final code example below. This might can take up to a minute to run, since we're processing a full video. "
"# Annotate our frame with detections.\n",
"annotated_frame = bounding_box.annotate(scene=frame.copy(), detections=detections)\n",
"\n",
"# Display the frame.\n",
"sv.plot_image(annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o8SsyCid6YV3"
},
"source": [
"Notice that we create a `box_annoator` variable by initalizing a [BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#boundingboxannotator). We can change the color and thickness, but for simplicity we keep the defaults. There are a ton of easy to use [annotators](https://supervision.roboflow.com/latest/annotators/) available in the Supervision package other than a bounding box that are fun to play with."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gKHSYO14vGC7"
},
"source": [
"## Saving Bounding Boxes to a Video\n",
"\n",
"Let's wrap up our code by utilizing a `VideoSink` to draw bounding boxes and save the resulting video. Take a peak at the final code example below. This can take a couple minutes deppending on your runtime and since since we're processing a full video. Feel free to skip ahead to see the resulting video."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"metadata": {
"id": "Es4SUdPGqlF-"
},
"outputs": [],
"source": [
"import supervision as sv\n",
"from supervision.assets import download_assets, VideoAssets\n",
"from inference.models.utils import get_roboflow_model\n",
"from tqdm import tqdm\n",
"\n",
"# Create a video_info object for use in the VideoSink.\n",
"video_info = sv.VideoInfo.from_video_path(video_path=path_to_video)\n",
"\n",
"if __name__ == \"__main__\":\n",
" # Download the video asset from Supervision assets.\n",
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
"# Create a VideoSink context manager to save our frames.\n",
"with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
"\n",
" # Load a yolov8 nano model from roboflow.\n",
" model = get_roboflow_model(\"yolov8n-640\")\n",
" # Iterate through frames yielded from the frame_generator.\n",
" for frame in tqdm(frame_generator, total=video_info.total_frames):\n",
"\n",
" # Initalize the bounding box frame annotator.\n",
" box_annotator = sv.BoundingBoxAnnotator()\n",
" # Run inference on our frame.\n",
" result = model.infer(frame)[0]\n",
"\n",
" # Create a frame generator object from video path.\n",
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
" # Parse the result into the detections data model.\n",
" detections = sv.Detections.from_inference(result)\n",
"\n",
" # Create a video info object from video path.\n",
" video_info = sv.VideoInfo.from_video_path(PATH_TO_VIDEO)\n",
" # Apply bounding box to detections on a copy of the frame.\n",
" annotated_frame = bounding_box.annotate(\n",
" scene=frame.copy(),\n",
" detections=detections\n",
" )\n",
"\n",
" # Use a VideoSink context manager for saving frames of a video.\n",
" with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
"\n",
" # Iterate through frames yielded from the frame_generator.\n",
" for frame in frame_generator:\n",
"\n",
" # Run inference on our frame.\n",
" result = model.infer(frame)[0]\n",
"\n",
" # Parse the result into the detections data model.\n",
" detections = sv.Detections.from_inference(result)\n",
"\n",
" # Apply bounding box to detections on a copy of the frame.\n",
" annotated_frame = box_annotator.annotate(\n",
" scene=frame.copy(), \n",
" detections=detections\n",
" )\n",
"\n",
" # Write the annotated frame to the video sink.\n",
" sink.write_frame(frame=annotated_frame)"
" # Write the annotated frame to the video sink.\n",
" sink.write_frame(frame=annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "EXTE3zWxqlF-"
},
"source": [
" Notice that we create a `box_annoator` variable by initalizing a [BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#boundingboxannotator). We can change the color and thickness, but for simplicity we keep the defaults. There are a ton of easy to use [annotators](https://supervision.roboflow.com/latest/annotators/) available in the Supervision package other than a bounding box that are fun to play with. Next, we create a `video_info` variable to pass information about the video to our `VideoSink`. The `VideoSink` is a cool little context manager that allows us to `write_frames()` to a video ouput file. Prior to writing the frame, we annotate a copy of it utilizing `box_annotator.annotate()`. Let's take a look at the resulting video. It will be installed locally and is called `output.mp4`. When run, you will see bounding boxes around the detections. Pretty awesome! "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We only scratched the surface of all of the customizable Annotators and additional features that Supervision and Inference have to offer. Stay tuned for more cookbooks on how to take advantge of them in your computer vision applications. Happy building! "
"<div style=\"display: flex; justify-content: center;\">\n",
" <video controls width=\"1000\">\n",
" <source\n",
" src=\"https://storage.googleapis.com/com-roboflow-marketing/supervision/cookbooks/annotate-video-with-detections.mp4\"\n",
" type=\"video/mp4\"\n",
" >\n",
" </video>\n",
"</div>\n",
"<br>\n",
"\n",
"\n",
"In the code above we've created a`video_info` variable to pass information about the video to our `VideoSink`. The `VideoSink` is a cool little context manager that allows us to `write_frames()` to a video ouput file. We're also optionally using `tqdm` to display a progress bar with a % complete. We only scratched the surface of all of the customizable Annotators and additional features that Supervision and Inference have to offer. Stay tuned for more cookbooks on how to take advantge of them in your computer vision applications. Happy building! \ud83d\ude80"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "V100",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
@@ -197,9 +392,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 0
}
+25 -3
View File
@@ -17,7 +17,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"metadata": {
"vscode": {
"languageId": "shellscript"
@@ -45,6 +45,7 @@
"source": [
"from supervision.assets import download_assets, VideoAssets\n",
"\n",
"# Download the a video of the subway.\n",
"path_to_video = download_assets(VideoAssets.SUBWAY)"
]
},
@@ -52,7 +53,28 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We're now equipt with a video asset from Supervision to run some experiments on! For more information on available video assets, visit the [Supervision API Reference](https://supervision.roboflow.com/latest/assets/#videoassets). Happy building!"
"With this, we now have a high quality video asset for use in demos. Let's take a look at what we downloaded. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div style=\"display: flex; justify-content: center;\">\n",
" <video controls width=\"320\" height=\"240\">\n",
" <source\n",
" src=\"https://media.roboflow.com/supervision/video-examples/subway.mp4\"\n",
" type=\"video/mp4\"\n",
" >\n",
" </video>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We're now equipt with a video asset from Supervision to run some experiments on! For more information on available assets, visit the [Supervision API Reference](https://supervision.roboflow.com/latest/assets). Happy building!"
]
}
],
@@ -72,7 +94,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.7"
"version": "3.11.8"
}
},
"nbformat": 4,
@@ -0,0 +1,566 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"gpuType": "T4"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"source": [
"# Evaluating Alignment of Text-to-image Diffusion Models\n",
"\n",
"---\n",
"\n",
"[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/evaluating-alignment-of-text-to-image-diffusion-models.ipynb)\n",
"\n",
"Click the `Open in Colab` button to run the cookbook on Google Colab."
],
"metadata": {
"id": "pviK1VFFNp8h"
}
},
{
"cell_type": "markdown",
"source": [
"## Introduction\n",
"\n",
"It is a common scenario to evaluate text-to-image models for its alignment to the prompt. One way to test it is to use a set of prompts, consisting of number of objects and their basic physical properties (e.g. color), to generate images and manually evaluate the results. This process can be greatly improved using object detection models."
],
"metadata": {
"id": "FtZXoFjPt569"
}
},
{
"cell_type": "markdown",
"source": [
"## Before you start\n",
"\n",
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
],
"metadata": {
"id": "1yOkAwH-UYPM"
}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MVfLwWd8D1ne",
"outputId": "70562805-224c-41ad-cf46-f6808f469923"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Thu Feb 29 18:16:26 2024 \n",
"+---------------------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
"|-----------------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|=========================================+======================+======================|\n",
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 46C P8 9W / 70W | 0MiB / 15360MiB | 0% Default |\n",
"| | | N/A |\n",
"+-----------------------------------------+----------------------+----------------------+\n",
" \n",
"+---------------------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=======================================================================================|\n",
"| No running processes found |\n",
"+---------------------------------------------------------------------------------------+\n"
]
}
],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "markdown",
"source": [
"## Install required packages\n",
"\n",
"In this cookbook, we'll leverage the following Python packages:\n",
"* [diffusers](https://huggingface.co/docs/diffusers/en/index) for image generation pipelines,\n",
"* [inference](https://inference.roboflow.com/) for running object detection,\n",
"* [supervision](https://supervision.roboflow.com/) for visualizing detections."
],
"metadata": {
"id": "xg5JFGa4UjZq"
}
},
{
"cell_type": "code",
"source": [
"!pip install -q torch diffusers accelerate inference-gpu[yolo-world] dill git+https://github.com/openai/CLIP.git supervision==0.19.0rc5"
],
"metadata": {
"id": "ri8GPZ3SD7LT"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Imports"
],
"metadata": {
"id": "U0qBpU5YU3P9"
}
},
{
"cell_type": "code",
"source": [
"import itertools\n",
"import cv2\n",
"from diffusers import StableDiffusionXLPipeline\n",
"import numpy as np\n",
"from PIL import Image\n",
"import supervision as sv\n",
"import torch\n",
"from inference.models import YOLOWorld"
],
"metadata": {
"id": "sBKUQMdwJTv1"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Generating an image\n",
"\n",
"We'll use [SDXL](https://huggingface.co/docs/diffusers/using-diffusers/sdxl) model to generate our image. Let's initialize our pipeline first:"
],
"metadata": {
"id": "ZzEhndAyU_EF"
}
},
{
"cell_type": "code",
"source": [
"pipeline = StableDiffusionXLPipeline.from_pretrained(\n",
" \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
" torch_dtype=torch.float16,\n",
" variant=\"fp16\",\n",
" use_safetensors=True,\n",
").to(\"cuda\")"
],
"metadata": {
"id": "qCwxwGcuEHJ-"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"In this example, we'll focus on generating an image of a black cat playing with a blue ball next to a parked white car. We don't care about the aesthetic aspect of the image."
],
"metadata": {
"id": "XrQ8ose0Vm-T"
}
},
{
"cell_type": "code",
"source": [
"PROMPT = \"a black cat playing with a blue ball next to a parked white car, wide angle, photorealistic\"\n",
"NEGATIVE_PROMPT = \"low quality, blurred, text, illustration\"\n",
"WIDTH, HEIGHT = 1024, 768\n",
"SEED = 9213799\n",
"\n",
"image = pipeline(\n",
" prompt=PROMPT,\n",
" negative_prompt=NEGATIVE_PROMPT,\n",
" generator=torch.manual_seed(SEED),\n",
" width=WIDTH,\n",
" height=HEIGHT,\n",
").images[0]\n",
"image"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 817,
"referenced_widgets": [
"80345cfe8a724651a287a20078218742",
"b79d4e509fa54f899391ff0ca80730c3",
"16cbb7e6ec6d444e8d71042afaf98806",
"4e3ebed950c945a0a7d74acaf8a4e071",
"0c2fc95f4da4409d8789c1086efc7487",
"a677a168d3dc46f48a6bd9877bf2873a",
"df3cb55850ba40afb692bca7f7557bbf",
"abb0cd4b86ed4d1bbb5bede80355be74",
"b403cba7f1ff4cd0a4a850de46bc7d3e",
"40ee234278f0466587191689f3a4e77d",
"48df83b840204738a2530fc222a17dc9"
]
},
"id": "TaWaIpjQERtf",
"outputId": "236e6107-b50f-4eb8-8db8-234519245264"
},
"execution_count": 41,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/50 [00:00<?, ?it/s]"
],
"application/vnd.jupyter.widget-view+json": {
"version_major": 2,
"version_minor": 0,
"model_id": "80345cfe8a724651a287a20078218742"
}
},
"metadata": {}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<PIL.Image.Image image mode=RGB size=1024x768>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaAAEAAElEQVR4nIz97ZbkyI4kCAo+lDT3yLzV1T3vPO+xv/bJpuvsnpmum5nhbqQCkP0BJc0ib/U561U3w8PDjR+qSqWIABDI//n/+n8bK0L+TINvNjbOp3z9c5ena5iUaVYVVczMzCHCLMlQFewfZTvHR8og9lnbmfsUx/ZhqlJUYrPcDf/j9/2H01z/OvGfU38+q8qiNKlFQISEgEIBACEAkiBUBYQISIgISRERQFSqSgACIoL+BlARkqpSZP8+BEJABICs31p/kFBdxyTRX30uXecCCChY7AMI1s9JiIKE4NcvQlRI3idBX/n6xz490fcFkOzf78vs3xK5r/TtYwTWOLzOtsYEb9dzjRLe7qiu44tgnes+vgCvT4EFUbBPep+2J+g6L18z0ud9G6s1zuCv1/86fv9+rYG75o6gXONwTwOuY75fSU8j1/fXyIgCxR6fexJf47MuCmtkrisUFfT4FyCv8RBd90ugZ1xV+ziyrr9EIBBTAakKEKZ9bKiKAMXqzwLoRV5FUe0r65m65vRaXQIWe81DeF0RVAVF0XXzAmivSVBUBQBr3a2ARUqPhmQVi2paxagKxqyzIpDTXQw2oCg9U448M/9C/eF8Mr5+fv1T4xgZkVrjQ8dmSMb3PP55fv2v8/yOYiTreM7nsyKzahvugorMyoj6+fNI237/t/9mAmR+/fnP4zh8c3VXFSkO16o8jnMGv8+apb9/Pn5/bBKnoGTo1znPMxQ5XImy4YDETBECGRHzCBEb2xDgODMTYxhZ5zGhIioZOSch2lNWmSJSlWrKQmWZq5kJxFwFklkEWTVcqgoQcy+yCgL4UBbnGVWlCh/GLB+eVaYmppUVmSpw13lGZomZmw4fAlGTiIg5zf08Yt+HDwOKZEZs+y6qERlR/YyN4cMVpI8xhqtKJb+/Dxv6/dchpmOMMWzOeD5PVVFVFenlFHOqiftQwXkcP38eahZJioqImwhobiTjDBv6+NhNZc4kxM1JEohIVXnsw0wyEyx3J3GeUVnu6m5qSuLPf34BfHzsH5+fUMzjMJN5HNtmHz/24T7P+XwepmP7+DC35/cZNZkVM8z0x49PQo5jzllZgOjYx/PrO+dhDhAfj8238fn5cJF5HvOc2+ePfd/nc37+2M/j/Pn1rSqbj23fnzP/+PPrz7++zjl//+1zG87ijEpC1FnMrDjPeU5zFZHM7Kdsc1NXEFWMYBazSkTcXBWZNWeIwt3dDSLnmWM4q8y01haBzBQws8bmPSmRkWeqqZjuHw8QzIg51VUhM8rc+9GOGcfzaULffBtuZlUVUQVRFVNRETUV0aqKCFMjpDJMZXvsxayIijQVE5qZqANynlNUWaVCHzYee0/u9/choEIEULOCZFZGQjDcWJEZBKqkCttjF1Cqqso3z6gsokiUuldkZYnKtjuqFLJ97AQFECCjsHZOg1oRc86McLfK2jYrcp6RmWJqKm7mrqojCTVVkcoQUEWqmJXzDDH1bYAUiKiokhEA1C2iqrgPH8NFDarMGpsLhChWZkZlqDkpSani58emInNGRpo7IBSSVAUzK2O4Pj4e+/ZRIoCYgBm+bRDM7+e2mQ8bu8ecrBoGFGKGbWPbR2bEOUFsY/dt2NgiA6iIaSKiKqS7bfu2+aAwK8/nt1R9fn7s+6f6OJ9nMcY28jzVhAyF2/a5f3ya2fdff53HT1URIUQp5vteRRbzfA73KorKGAaBqoGq/sP3ByGM4/z6z+fzCcmqBFhFc62EuZlID+7j40PVquJ8Tt+H2jAfY8jx9T3PQ03NrZisMnG1jTQ1A3K4i6raNvYfInJ+PwUFVBEi2pMCiO8fmYiSKsb5FFblCREoTKXfpGP7UB3mHyKGYkYRpSpkkFVVZgPQ8djmmaogCwWIQrWfEZIsiIgYVExg1CEKRqhaJYqaSVUjiWJE/vz5/TwOkI/HY9sfPkbDTyyQQBVlo0ws6LLAwwuiEUQBrAJvhHZBmgtXiWrDlv7LBeEaBTeEoDTSAwGqClmQEoQos6ZKKEJdRLVYcU6B2diSyCKzWCBBF5BMGiSlPDMdglnbNrg/4I9CyuaZQBUZgROosQ9zs00KqJlAqrsaymUyVYxSAkLuK1NRRZSkkDKfwEOH2gCUJSSrAF1gF1wATQgutC0CAVgl0Av5NcosAqjrYxcOXjASgKB44+01Ezfyu5DXwnovnoAbIwIv9AaQUoIb9/c1yAVfX5dwXwqamVxYVShrmuUCdIS8OMlCt3LjwRs14nVRr28ad143upAwANH7BiAvpnNh8b5irt+vi0fdV/86jqyL5XUUuRA/rjGV60qukcfbwbgI3H0FJNBL4h3Iv87bT5Ss4X5d0MVh1oX0La9HDRTgOuT9g3secR+JVTe97Iu65rH/lRCg3sZ3zeBrZa3HL1NE15GLr/VwsSkBqmiqAFnkPbR8TeP1+3hbb+88CbV4Zt/POoXcbGbxBABkQd9ozeI5/TpcWxJJmlgi+xfENKcUlCqcxRSKylBR18avX1+m+vw6lJGV8/mdNc0eqqLDas7j+c15zmNCODbnmUfOWVGRNiyZGVWZTFQVlclZNc3sPM4kt48hqlQBZc5gpZokYa4/3G177PtjqPJJcNrwgSIKxWCA5GRGicj+GGYGMCdlsVsBoQpIZWQDE8mGKyVKpuQMkjZ0cXNQFSKMOQFVczNTNQJVCdYi52DTucyqZFVBoQIRYZWoZGZElpXDIKIqKoioIiFkJUxyBgCjVVURCqoBAhGKSPOQYiGaVbKKVaUqQAllf+i2baoyz7ltPiPJYgEogYnctNPcVV2P86zJbYx9H5nFqfvHdpwJxTCNrMgUkEIf7puLgqwsAMxIdzXVqH7NqCpEpKoywoeLCgRqQjIrj3nOIyE016qcxyHDkgVCXfaPse27qwH551/niZDhouquOSXBQikkMgWakb13kTXPMyIqwnxAuD3Gjx+/DR/ktPF4Ps+IQMlwL9gx+f0dY1cQ6iYm6mKuvz8+x7CcMc8ooErMAUhm9Eu0H6tKQvB4jH3fVCQykSUVeQYAGy5SmYRAtDduZmY/91VZLFRlcIwhqqqaMauqWExko1KTqgLr+H6aKVDFYkA3H7uZaWMCH9u2GzN7r8jMIokS0cxSsaoCkZyZZW5QkEVhkjNDVKBC4Rn5+bE9Pj5V7OfX95zTh1cVlIRUZUMEkUtKSNQlvajJGC7C53fNmWPYGLZtQ21kVsahgsrKzIhSU3eHSOtZUBBi6gLGjCqayba5bkNU8gxIq1HFKgBVZS4AkIWLLUQki2qj5rl97L6NeSZZGdnkhxAxQatsJMFhjuKFpVns5azFMjEB1cRc53GqKURYpaaiso0RlDiTZLLmGRFhxbENc61Mc0lIJWGqZmLYxjbnnHOa0obY8IpTTYqckTGDoLuLcntsjx8fCZ5fMTNVNaQq8uMhrvb8ms/vw10en4/znD4kI2t4VUVWZqmWuo3NoOL78G0/ns+v53OM8dhdVWeEV+2Px9j9PPD9PMcw3yUrJNTH5h/jlDi+TvMx3GAK4vvrcPfN52P/XU3PZ1SYnnrGFFOyYEiSKu6qpjnposWWmMRcIGLu/bYqUNRElYCZz5xRfOzGEh2m0PM4zbdNAUpRi9KrvWbs+1DXSkZUfD8B9W0f264o1pxQCCCYkSz68CqoqpmrjTzmcBOVzIiqKvrYAKmSeSwBpV+kaipqgcqs9b5UYTEyzcVcCVT1O9QYFNEqgmCicu0TEL0Bg9jSfVkLLYlIY9KFROpdUL7e4wD50jf5Djt5wQ65RL8Lwq23zMJBuCjBgnGqEG0u9RLBIVJgFSHa9wUxCksqIwj1bWOyKnSAKY55zoKYu2YhRKZuEDpOq29WTJNQh6GMKSdVVAkfBpbWtHKAWQmpEoQqxJLBREFNQEFW/XGck/IbXVX3TU9aHmQGZPC+MTQduMcIApj6el3frEreEFEDwQZqvPDmUqZfavuFTtc5LkR2//SeqkXOljp/w94L7t9xgxsbLtDcIPdXsfmlKBNUuU7baPcViFhxgLej89aFF7AWyL2krtu6OEx/9l4fd+SAeA3QC4L2/93MYI3m+sc30Po2Wq3DvziMyHWP/ZNGwK91fJ2Ka0rkmi9Zs/Uav+tCb9h+xXkWw+E1v41rLywOuf9HvF/nC1h3yIW6Ai8vdvMiHtcSaHrSswARroUmTQ7k5klrumQ90tcpGuUJ4ap94XIfdq1iFimitXaNehGV3kJ6zwBEtMkW+es81FuwAItGCgDUCpLICgvcmwBeS7tUpVhFzmQEk4hjSqYAvo9S1RIIlVRKHpPfz8yfn6PmsPPrp6hKHvWsipk5cwarxHWMTSSeXyZiYohIFZ3PmZkmum2+79tmNobsYwzV7SHHcRTEzBpyxvnUgoqomo1t7I9t3w0VZefzzDlNxM0IQXHb3Fy/f36riYqOsZkPs/P76yhWZWVvhKH9furQh4iYaRWI6sc1swQKXMpiUaQIRiQIM73WjxLMCklVs95z54x+pNzcXfvJIzvAUpViw81URcgpIqpW1dMqkVlRVcxMEYwxtm34ZgJGyJwzzrg2cQHQEadMmtl5ho1UamQWeZ5nkQrOGZEpIr5ZzmQF6TErIwHMGWIGytge5jnjK85MZGWJUFRJVNJUI2OxfEhrsWaaWaoqfRyRrFLTpbwOFyAj5zyvmAkrKEOSNb8mkdvmH499ezx+/P6bkPOPJ1TOc+pxzEhzPc+ZkQ2mzzmHe1ZUrHliQkRsDIjuu7v74/NhqnHmX399USiC/WP7/Pgc2+O33//x47c///rrn5nz6/ktOkB8fu4qKiIzGZWkVKGK5IpsqCrRATcVFXc3s23fe+ng6xmRVSTrOEtF1K1hdLP0Zg/nebIRn7uoRmZGRISIxMyxGaBmCoFkVVVlghSFqJoKKyBSpLine truncated
},
"metadata": {},
"execution_count": 41
}
]
},
{
"cell_type": "markdown",
"source": [
"Not bad! The results seem to be well-aligned with the prompt.\n",
"\n",
"## Detecting objects\n",
"\n",
"Now, let's see how can we detect the objects automatically. For this, we'll use [YOLO-World](https://inference.roboflow.com/) model from `inference` library."
],
"metadata": {
"id": "wKuyU-GDXPhr"
}
},
{
"cell_type": "code",
"source": [
"model = YOLOWorld(model_id=\"yolo_world/l\")"
],
"metadata": {
"id": "R9pERjDkGIlN"
},
"execution_count": 42,
"outputs": []
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "G0PlnmZckOCH"
}
},
{
"cell_type": "markdown",
"source": [
"YOLO-World model allows us to [define our own set of labels](https://docs.ultralytics.com/models/yolo-world/#set-prompts). Let's create it by combining lists of pre-defined colors and objects."
],
"metadata": {
"id": "VOxtES_qaoWe"
}
},
{
"cell_type": "code",
"source": [
"COLORS = [\"green\", \"yellow\", \"black\", \"blue\", \"red\", \"white\", \"orange\"]\n",
"OBJECTS = [\"car\", \"cat\", \"ball\", \"dog\", \"tree\", \"house\", \"person\"]\n",
"CLASSES = [f\"{color} {obj}\" for color, obj in itertools.product(COLORS, OBJECTS)]\n",
"print(\"Number of labels:\", len(CLASSES))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "SkY3TI4JHmkj",
"outputId": "253a95a8-0c32-403a-db63-82e529de2eba"
},
"execution_count": 43,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Number of labels: 49\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Let's feed these labels into our model:"
],
"metadata": {
"id": "by_NYnqkoET-"
}
},
{
"cell_type": "code",
"source": [
"model.set_classes(CLASSES)"
],
"metadata": {
"id": "oinXZU3YWZqw"
},
"execution_count": 44,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Time to detect some objects!"
],
"metadata": {
"id": "RTHHpRsFoJsk"
}
},
{
"cell_type": "code",
"source": [
"results = model.infer(image)"
],
"metadata": {
"id": "zDPd-LOqGLHJ"
},
"execution_count": 45,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"We'll convert the results to the [sv.Detections](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections) format to enable features like filtering or annotations."
],
"metadata": {
"id": "TGCRxUpeu8Vi"
}
},
{
"cell_type": "code",
"source": [
"detections = sv.Detections.from_inference(results)"
],
"metadata": {
"id": "GWbmFKscvUhU"
},
"execution_count": 46,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Speaking of which: we only care about strong detections, so we filter out ones that are below 0.6 confidence."
],
"metadata": {
"id": "AscEY8SioQQ7"
}
},
{
"cell_type": "code",
"source": [
"valid_detections = detections[detections.confidence >= 0.6]"
],
"metadata": {
"id": "f2fvqc8lWxGT"
},
"execution_count": 47,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"A quick peek on the detected labels and their score:"
],
"metadata": {
"id": "klstslXPop-9"
}
},
{
"cell_type": "code",
"source": [
"labels = [\n",
" f\"{CLASSES[class_id]} {confidence:0.2f}\"\n",
" for class_id, confidence\n",
" in zip(valid_detections.class_id, valid_detections.confidence)\n",
"]\n",
"labels"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0jbyOEGdGOhd",
"outputId": "a7a90eb0-d829-40d3-8090-151e2c735d8c"
},
"execution_count": 48,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"['blue ball 0.95', 'black cat 0.72', 'white car 0.68']"
]
},
"metadata": {},
"execution_count": 48
}
]
},
{
"cell_type": "markdown",
"source": [
"## Visualizing results\n",
"\n",
"Now, let's use the power of `supervision` to visualize them. Our output image is in [Pillow](https://pillow.readthedocs.io/en/stable/index.html) format, but annotators can accept the image to be a BGR `np.ndarray` or pillow's `PIL.Image.Image`.\n",
"\n",
"Time to define how we want our detections to be visualized. A combination of [sv.BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [sv.LabelAnnotator](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.LabelAnnotator) should be perfect."
],
"metadata": {
"id": "GOccbqMDo2Ec"
}
},
{
"cell_type": "code",
"source": [
"bounding_box_annotator = sv.BoundingBoxAnnotator(thickness=2)\n",
"label_annotator = sv.LabelAnnotator(text_thickness=1, text_scale=0.5,text_color=sv.Color.BLACK)"
],
"metadata": {
"id": "X9HMMnTppp-c"
},
"execution_count": 49,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Finally, annotating our image is as simple as calling `annotate` methods from our annotators:"
],
"metadata": {
"id": "u3y59TgWp4Fx"
}
},
{
"cell_type": "code",
"source": [
"annotated_image = bounding_box_annotator.annotate(image, valid_detections)\n",
"annotated_image = label_annotator.annotate(annotated_image, valid_detections, labels)\n",
"\n",
"sv.plot_image(annotated_image, (12, 12))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 734
},
"id": "UdDEwjhoIiHo",
"outputId": "c56bd631-653c-4fd6-cc41-653abc49b0e7"
},
"execution_count": 50,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAALNCAYAAAAbaP0uAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz925bkSK4tik3AzOgeWd1rrX2+Ro/6FP2MvkNP+jU9SPvs3d1VmeGkmQF6mICR7hFZlWufo6Gxxgj2qM4IDzppNNoFE5iYEHd3fB1fx9fxdXwdX8fX8XV8HV/H1/F1fB1fx3/RQ///3YCv4+v4Or6Or+Pr+Dq+jq/j6/g6vo6v4+v4P3J8Aduv4+v4Or6Or+Pr+Dq+jq/j6/g6vo6v47/08QVsv46v4+v4Or6Or+Pr+Dq+jq/j6/g6vo7/0scXsP06vo6v4+v4Or6Or+Pr+Dq+jq/j6/g6/ksfX8D26/g6vo6v4+v4Or6Or+Pr+Dq+jq/j6/gvfXwB26/j6/g6vo6v4+v4Or6Or+Pr+Dq+jq/jv/TxBWy/jq/j6/g6vo6v4+v4Or6Or+Pr+Dq+jv/Sxxew/Tq+jq/j6/g6vo6v4+v4Or6Or+Pr+Dr+Sx9fwPbr+Dq+jq/j6/g6vo6v4+v4Or6Or+Pr+C991F898f/+//h/ormgD0G/vcG3v8F9Ao9/oPR/QucDxToEByCO7b6hbRukFsw5gTFQVVHaHVbf0NHgusFwx8PuOHCH6Q1SGxQKmY6bGDYV/Ptbxb/fFaUo/jEc/3g4Hh0YXjBdYRAAAjj4HwB4/CCAxO8CPX+XyzkAIALA4QBEBALhz0+94Dzt5VOJvz19Lq8/OCTbuc7Ptr1cUgTys8u9HO6X64hc/8AP87OzO+Aiz8/+cn2HXz6XpxPysp83SP60rR+vfl6Tzf/zb35+8L1dn/35WS7XfnpmWS356V3/oj1x50/b/affXM+b34+Pn8Zj/OPnV85x+/xOPtxTZJ0rL18WubxICERymOTTACqAPv0NKBqjVy5tdLbZ454OwDynlsc1X9/GOf5VruPFY96dzyKCsw0IL5yc56/z4ieHw9z4swMW/TkAPEZHtw6fB/x4YCuKWhq20jC9Yp+Ox+OfEPufGD/+B8Te8dj/BX//A8UmSvsN8vY3QID+/Z+Y+79wfP+fcOmQWrE/Or7//k+MxwG4Q4tApsPmgE/2x/txwErD3//2bxB37D/eYdbZ4UWhLhh7R1GHKHD0CRGF1A3b7Q1brcD+gM8DuhU8+oF+dLhNuE+oCmxyEdxuDbUq9v3A8Rh8r6qYBsxhkCIwM4w+OJwcmNMwhgOicHPYnJDCQap6jhtRgZtDtWDbKlTpHzU3iBvcDOaOWitcBDbP92VugBtEBCocd+6OOQ2lKrSUc7SYY84Jj3faaoVAOV9EMOdgW9zRakNr/JvNGXM72mWOOR1mhloLShGoKH777Rtu9w1wx7F3HEfH47HDHLjdGlqt6GPiODpUFbVWlKoYNnE8drRWcLvfYNPweOwYfWI/xhqV5gaBo1RFbRUwQBSole2aw6CloNYCM2CMiVoUt1uFiKAfB0SAtm0Y02Bj8soKjDnR98m5WRW1VNTWYHCIGBSOb982fPvtG4ooHo8/8I9//gEtG97efoOqoveO3jvG6CiquN/vgAv2vWMa4C5wEfTeMfuOthWoCv7933/Db7/9DbUUmB14/HhgTEdrN2y1Ybvf8ePHO/74/Q+UJthqQd02PPrA77+/Aw7UVuBj4tg7DIA73/00x+gTc/S1Lo1hgABvbxu2rcVnE2MMPB4dNh2lsm0WC9CcBlGgaIl3gRgrhqIck621NY7mGBBVlCJPK1Zer9QKVYH7hBvbLyoopcQYxjn2zNYaZe6xRrJtrRZInDw5MVBqBURgNqNvKkQFmOyHooK/ffuG23bDYz/w/Y8fa9yX4mitorQNDsd+dNg0qACYAFThwvkKcOzNObE/dogAtVZsW0OpDXMaZj/gbjEecx4XiCpsGuac0CLY2oYiAsBWX21bgSrbPo/BGa+FY/UYXPNVUIoC09DnhE2DFOXcLAVtqyilYLvf2IZjx+wDWhQiimmGaZPvKtadWgvXhDn4jgAcg2tR2xpabdBSIGDfjt6hRbnu9IPvsVaIFozha885HpwftTXUraIU7jECx+gdpSi+vd1xu91Q6obeO2wO1Kq4vd1hYfeKCqQIZh9wON5uBQqOv3rbcPSOx493uBlqayjacP/tDTDD+4939GPH7dagqmhN0WrDdtvg4Dow9nfUKvi3v/8bbrc3mAtEBX0c+P7P31FLwf3eIKpwueHbb3+DCvD993/g/ccPqArqrcAhqPWGtt0gKti//4HZHXWrqFuBQHA8OkopuH37G/7+b/8bRByPH7/jxx//xPv7O0wMaYh7GDfb1rjXT0FrG0ptgDh8dkipqNsdtEUMfX/AneNEVbj+G3C7v8GhKLVB3DDHRG133O7foOWG3jvEByCGMSZu9xv3u8m1X6SgtA2l3TCPAzYP9P5Y9uGcEyKC1m6o9Y7t9htK3WDHpA0swJwdY3ZAuJe6C0S4X4lwXMABjX1vjnHa52nsGed72d7g7rAxIFDYFJhzHXHn3jf6wI/3B47jgJaC+/2G1m4cqznpXuy6C+x5+vtpCec27lzbApfYFTcF/oEAEnOMO9ozfsnve34Nzr3aOYdUHaIG9wGow71D0KGa9ofDpsFNos8KxjTMMWDDoLVCVDH7RKmATcf/5f/6f8OvHL8MbDEGugFSK6pOOA5occhNABTYEPiYUDWUKihu0DkAM1QBahUIDOIdxSoA5wIljioCk4KJAjOBgRPfAHQz/GM/0FHw7cYH3W6K7o7Rw1JwBBBdMGYZ4AFVoVKWUbwAxAsgeQVcnx25gX32vc+g8EfUmrdOoCUnejmtdl7zcq+Exa+tfAJVV8z8Amg/Au3XBvHcD33werrkIH4BV0+Y8RNo6a+nyfVxn9rwETl/1rCX73zS1E+b87PjySmAeCefv7v89M/GS57zOi55K//03M/bdZ7k2V+vXo8P9zz/vobC8kic130er77O5frrEAfKFWzGunZeyuN/sr6f9wQC3F5a53CI5ELpaxX2uEcCmPNVZH9dzo9NW+JGwv87/Th+tkTgmO4Y0zGH02jeO8QMqo6iG2bMQ4GhODD2AXvswHjH26YY2w398QPwARwPAIDZoLMOgGohoDCDaoGUAhuD9+sTow8oBNtWsd02SK1oTVG1oCqwH4JhToAjCrhjHA/AHCqKUiva7Ybb/YZaBBMF+w/H7B0KoJYCqECkorWCfnRMGyiq3ARLg+qOx/tBIDsm5nSol2XQTbMwJhSlOMwQ7yrfofK9qBAUmHOjgmPMieJ5bgBV0Eifk30Ct7gHX1opBa0VnAZQbI7mgNJoEBEYJsyE14AQ9AYIJhgmmN82Ah4aX47esYzM0/XhC4DP6UBxHL1DSvT5nDh6p4GjiuPo6GOwT6rCzeDOvhtjwMzQu0O0QESx3e4Q7Tg6n3vNBI0ZaXQOjDEWsDcHfEy4eTiC2L4xRjiJuBOOMaBaULYGm4YxCF4gAZwmoGoEoDZRquLbtxu2+x3ffvsGN8OPB9ePPjrweKDUghngMGdvHxw3c47Yh2WtCaU2iBDwl1Kw3RpUgP3h6JPG2bZV/O1vf8Pt/g3//h//Dd+//4F//vN/4jgemPsBc8FtqxBRqCqGA4aD480tHBAxFkWX8aUqAToLaiWAb+449gNzGjoG3A3HYVBVaC0otdA5Y77WtNkHjala0GqDCzDHYH+aQx0EQEInOgSQaXA3+Jww1wCzANzgk86zUirbXAgAPcCsiqAqnxXK9ygCiDmmGWwazA0OBLismGNizomCwvZIxXZr+O3v/4bfvv2G7f07zAkUa1G4cF6lgyqGG+aYsMnxqYXzl0CQxnMpBOq1arx/C2NUYQbQCwP4DAPWJox/wNZu+PbtjqoFvR84joN9JXTgCARTLdbBiWnsC/aNwSfXeY13KkpHxXWvy9+nTYw5UFEBBQ1h5xippaDVirpVqAh8ErxCBVomnRWxrwgU5o798cA0RzXO69Jq7Dl8T9um7DufMDFMNxQ45hiAcP1REbRtA3tO0ftAH8Z5CsBsQqJvIYbeDXacjtzjmHh7u6Ntd9Ra6QwwYN8f6EeHFQd+cA/bA+jRWSHsRwEeP96hpdDxUApghuMY2DY6K0opgAO3+w0wh7ugakW93dFawf540Kk1B3waDAXtdkNrdHSMftDRrA63iXEYPDygBB4dIpNgP96HKCDh0M5zixbAHBYGzBgTFue0Vjg3jA6TOQedQsL1iM41GsFjdGhp8DmBsDhqVTpCakOrir5P9NGxtQKJ/WZ2wzCjAwgb1AWlNhw21jwF6LhPQ0JB0D2dY7qosj0zxqfRgV60QpUOtRlrdilKm89z9YxdLmwhKYJSBLUVqBQMEfRjhBM+1461M/I78mpSnwG4ZYmdxt8yl8/vXOxNSbvp+fPnuI9jgeWriXgxeh2ru+L7Z196ttE9bC3QMFPEWDRY94XbJAJ50yZmj3krgNuEhdPeByA4nd5/dfwysLWywQRQcTR0YHCiwSdMBNAC14JyK5AicBhgA/RYC7ZCqMqXXNGEE4YTU+FaIFIxUDABTAgOp1e/T9qVswpaRA5KcdQJwAwK5XVFzhfuyjasTjdcI6GvUTaHQ5epn6DrNcqax/mG01t7Qc2fnHv+u2CAc7Q+Y47LKLrg0udBeEFxfjkx/nQa9TkbnkGh5zOt538e9Ofznp8v/H3pm/OGL33zAW/Jh3/88ijnI8jzh09fk0t/fAInP0Gu13m+PnkFTK+NfXJ4/OS5/OMjPnnlnvru/Pn6/3/S7PMPl0VCRD4sbFdnyDMQfLnHpSMFkqsOMsIqOBfdj69SFnBdkdmzVZc+icU4Xd1P4zKfw+FCyCxPb4WANefjZak+nUjX8RC3ULine truncated
},
"metadata": {}
}
]
},
{
"cell_type": "markdown",
"source": [
"## Testing it automatically\n",
"\n",
"We can also test if all requested objects are in the generated image by comparing a set of ground-truth labels with predicted ones:"
],
"metadata": {
"id": "kl5kO59CquMq"
}
},
{
"cell_type": "code",
"source": [
"GROUND_TRUTH = {\"black cat\", \"blue ball\", \"white car\"}\n",
"prediction = {CLASSES[class_id] for class_id in valid_detections.class_id}\n",
"\n",
"prediction.issubset(GROUND_TRUTH)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "laJS1E76qynK",
"outputId": "de193606-b41a-445d-8107-de02218190bd"
},
"execution_count": 51,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"True"
]
},
"metadata": {},
"execution_count": 51
}
]
},
{
"cell_type": "markdown",
"source": [
"Using [sv.Detections](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections) makes it super easy to do."
],
"metadata": {
"id": "8L4lXpvFvxZR"
}
},
{
"cell_type": "markdown",
"source": [
"## Next steps\n",
"\n",
"In this tutorial you learned how to detect and visualize objects for a simple image generation evaluation study.\n",
"\n",
"Having a pipeline capable of evaluating a single image, the natural next step should be to run it on a set of pre-defined scenarios and calculate metrics."
],
"metadata": {
"id": "j2GgYtNGqYmY"
}
}
]
}
+222 -55
View File
@@ -2,7 +2,9 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "BsraWznGULVc"
},
"source": [
"# Object Tracking\n",
"\n",
@@ -14,8 +16,71 @@
"\n",
"## What is a Tracker?\n",
"\n",
"Trackers are a piece of code that identifies objects across frames and assigns them a unique `tracker_id`. There are a few popular trackers at the time of writing this including ByteTrack and Bot-SORT. Supervision makes using trackers a breeze and comes with ByteTrack built-in. But first, let's get our deppendencies installed. \n",
"Trackers are a piece of code that identifies objects across frames and assigns them a unique `tracker_id`. There are a few popular trackers at the time of writing this including ByteTrack and Bot-SORT. Supervision makes using trackers a breeze and comes with ByteTrack built-in."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7h__vutkUZSY"
},
"source": [
"## Before you start\n",
"\n",
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "pzm2gQlYUbmd",
"outputId": "71217cbf-f7d0-4496-c150-1c363386a16f",
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Fri Feb 23 03:18:02 2024 \n",
"+---------------------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
"|-----------------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|=========================================+======================+======================|\n",
"| 0 Tesla V100-SXM2-16GB Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 33C P0 24W / 300W | 0MiB / 16384MiB | 0% Default |\n",
"| | | N/A |\n",
"+-----------------------------------------+----------------------+----------------------+\n",
" \n",
"+---------------------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=======================================================================================|\n",
"| No running processes found |\n",
"+---------------------------------------------------------------------------------------+\n"
]
}
],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZI4EwAnEUTFC"
},
"source": [
"## Install Dependencies"
]
},
@@ -23,18 +88,18 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "powershell"
}
"id": "NaEjHlAWULVd"
},
"outputs": [],
"source": [
"!pip install -q inference \"supervision[assets]\""
"!pip install -q inference-gpu \"supervision[assets]\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "zUKICBWFULVe"
},
"source": [
"## Download a Video Asset\n",
"\n",
@@ -44,94 +109,196 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"metadata": {
"id": "MyNzGpzOULVe"
},
"outputs": [],
"source": [
"from supervision.assets import download_assets, VideoAssets\n",
"\n",
"# Download a supervision video asset \n",
"# Download a supervision video asset\n",
"path_to_video = download_assets(VideoAssets.PEOPLE_WALKING)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "0cMveUy5U-tN"
},
"source": [
"## Tracking Objects\n",
"<div style=\"display: flex; justify-content: center;\">\n",
" <video controls width=\"1000\"\">\n",
" <source\n",
" src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\"\n",
" type=\"video/mp4\"\n",
" >\n",
" </video>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lowrxifpULVe"
},
"source": [
"## Tracking Objects in a Frame\n",
"\n",
"Now that we have our video installed, let's get to work on tracking objects. We'll first pull in a model from roboflow Inference to detect people in our video. Then let's create a `byte_tracker` object that we'll pass our detections to. This will give us a `tracker_id`. We'll then utilize that tracker id to label our detections with a `label_annotator` to display the tracker id. Finally, we'll use a utility called `VideoSink` to save the annotated frames to a video. Let's dive in to the code."
"Now that we have our video installed, let's get to work on tracking objects. We'll first pull in a model from roboflow Inference to detect people in our video. Then let's create a `byte_tracker` object that we'll pass our detections to. This will give us a `tracker_id`. We'll then utilize that tracker id to label our detections with a `label_annotator` to display the tracker id."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "gtbnWeT8ULVe",
"outputId": "0fbc468d-ea0d-4d23-a44f-1d50e37f95bf"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9Z5BtyXUeCn6Zex9T59Qpe+uaut63B9qi0bCEIQmSoGhEmSDFCSr0Y2KkCEUoNKEfMz9GPxUx789Ib16E3rwgFCRECSDBAQQSIIAGugl0A93o7tvXtLveVNWtqlveHLv3zvmRe+VemTv3qVNN8s1jRK0bdc8526RZuXK5XLlSKKUU9mAP9mAP9mAP9mAP9mAP9mAP9mAP/p6C/P93A/ZgD/ZgD/ZgD/ZgD/ZgD/ZgD/ZgD/4msGfY7sEe7MEe7MEe7MEe7MEe7MEe7MHfa9gzbPdgD/ZgD/ZgD/ZgD/ZgD/ZgD/bg7zXsGbZ7sAd7sAd7sAd7sAd7sAd7sAd78Pca9gzbPdiDPdiDPdiDPdiDPdiDPdiDPfh7DXuG7R7swR7swR7swR7swR7swR7swR78vYY9w3YP9mAP9mAP9mAP9mAP9mAP9mAP/l7DnmG7B3uwB3uwB3uwB3uwB3uwB3uwB3+vYc+w3YM92IM92IM92IM92IM92IM92IO/1xAO+uC///f/vu99IYT1WQRKKesd+pNSolQqQUqZu9+vnDiOEUUR4jiGUsoqk7/vlkV10rUwDBGGYe4ZX71CCCilzB8ASCnxJ3/yJ7h8+TKefvppHD16FBMTExgaGjL18Hc3NzcRRRE2NzcxOzuL6elpfPSjH0WpVDLPSSktfFD9vN4kSXJ92c048LL4O3TdxSO/F8cxkiSx3k+SBL1eD3EcmzFJkgQXL17E97//fVSrVUxNTeEzn/kMqtUq7t27h4sXL2JxcRFBEEBKacorl8v4xV/8RXz0ox9FFEVWW3i9ABAEAcIwxH/9r/8VV69exfT0NEZGRtBqtRCGISqVCqIoQq/XQ7vdRrvdxtraGoIgwNjYGP71v/7XGB8fR5IkFn24+HdxSM+XSiUEQWC1aaexIBwCGf2VSiXvO0mSIIoiUyene+p/EARWnXxuSSkRBIGhP/4M4dalIV/7Od3w++Vy2fSfnvc96/af44DmP+HU97z7qZRCr9dDkiQIw/7srF97hBCoVCp2X6Gg4G8LBxobl3/46nav8fZXKpXCZ/q1nehACIGhoSHMzc3hj/7oj0y5RJsAMDw8jF//9V/HxMSEt328fKUUyuUy/vRP/xR/+Zd/iVKphC9+8Yv4+Mc/jrGxMfyn//SfcP36dZw5cwb/6l/9K1QqFcOP+vFuF2g+B0Fg6L8I3Pbx79RX3/0iSJIEc3NzWF9fx9LSEu7du4cDBw6gXq+jVCp552GSJNi3bx9OnTpl1RXHMcrlssUz+NgV9Z2A5gHRAR8TzvcIaF7z5zqdDmq1Wk4e83lfRIsA0Ol0jCwedPw4EF+iuej23y3Th99er4dyuVwof11w5T3JnVKphECGECiek27b6HcURUamuM/yOenyKi7r4zhGtVot7OtO13u9HpRShm/z8otw4ZbX7XYNbyT8KCgACi5aeLkcDwDM+67s4Nd4m/i1brdrypBSIhBh7hmCRCU5nhtFkRmPcrlcSAM0HhcuXMB//s//GUmS4NChQ/jN3/xN/P7v/58wMjyC96++hz/6oz/C4cOH0Wg0UKlUcOvWLZw/fx4jIyNIksTMX7cepZSRlcRn+tF2EZBuxPHo1uP+9uHYHQPSx9x7Lg/wlc/bXzTvSOdSShm+49bpvp8sTKH3g08iR2wAwirw5O9JvP3VBHEEPPSrAuuzQFgGTn1W4NX/V4LOpvOSjFH6te8D9a1c++I4hlobBb7/Syhasxs5CmzdB5IIEBIYPQKszwIq9j6u58rnvguMr3p5qg9P9OnjL0Rbg+jnxEv5+/y+awdwXhTHseEf/F6328VLL72Ey5cvG3lCf0VyIY5jnDt3Dv/4H/9jM+6+OSKlzOlALg54Xf1sDyEEwjA0vI/+/i//53/ZF28EAxu2OwE32oqEl+8eMWoS0lxQFxlqHCncCHSVKvd9H2ERAvsNSBHj4QNWqVTwqU99CpcvX8Zrr72GCxcuIAxDDA8PY2hoCKVSCe12G1EUodlsotVq4ejRo3jqqacwOjpqmL7LiHYCjj8X5/0MDFcg+XBV1H9+n48pF77cYBFC4MyZM+h0OhgeHsbY2BiklJibm8Nrr72G7e1tVCoVTExMQEqJxcXFHCP3tZW3q1qtYnl5Gc1m0yilgDb4er2eZdjRmAPA2NgYkiTBwsICJicnrTq4YdtPmHIFqAiKDMYoiizDk3BXJGy4MUhChRusLv37lJVBFP5Bgde5kxLre5f+yPAqErr8OxeyZOS7fGO3CsdOBlW/PpDD4cO+zwXFTs/yT18ZSZKgXC6jXC4jDENLqJHSf/fuXTMH+5VPuO12u6afMzMzUEobvCdOnMCNGzfwwQcf4Oc//zk++9nPmmepPTspbbweer7IscH7yn/TO/2Utn7lcQUiDEN0Oh3jZHF5MuGR172b+gahxZ2M0J3qc/nAoO/Rs3EcWw7WQWib44TkMK+HGwyDlEN8bdDneV30XSlVaNT2K494P3ca+viJDz/8OzlSudzZCVzeT3OBeORORi2/x/th6RUQUKK/s4WeLZVKufnp4sF1Rrp9CILAOBugUmMBeT5d1Bdy2nIHnmuskRwlnWJrawulUgm9Xg9vv/02xsbG8NnPfBY/ffWn+NGPfoRWq4UkUXj66afwpS99CRMTE9Z8LqJ9Vz5zPOw0NnSf6Klo7rq/i57j85zK9+m3O4Hv2UHojNdZpK8MUj8AJD3gxosKhz4qUKoBm/NArz3Qq1Y7CuxTAxv3su8qAdbuDl5HEfC+c5pxaZW3dSeg/gRB0FceUv2ubkS8i5dFjt8XXngBd+7cwdraWs7ALIIDBw6gWq16dZ1+8sb329VJqAxe3t8U/tYM20HAxygI6T5mxZ/h312r3zVCXOBGs1sW9woAxcrKTkw4jmOcP38eH//4x/H973/fXGs2mwiCAIcOHcLMzIwl4Hu9Hk6cOIF2u42RkZFdKdcuM+MM10dggzBM3keXgRdNXp9BTE4CWskC9Orro48+iq2tLSPwh4eHcezYMYRhiJGRETQaDSwtLWF5edliCjTWvC4Slrze69evY21tzWpPuVxGpVJBu922lBZS/icnJ9Hr9XD9+nU8+uijlkeK00uRYsKFidvGIseAOwaVSsUYtW45RULNZV68PbxsPidIOXHLct/zldWP/l3j38VD0Xyi73yVw21D0TXCATHIQZVm3/zgePLNf7cPrhAZpP4i/NH7gyjxVGeRUkXOlXq9bpxIvV4PnU7HPBPHMe7cuYOHH34Y1Wo1V5Zr1G5ubmJpackoyLQCBGhnEtHg1atX8fzzz5s27jRvfHggA7Pfir1LY1yJAPJ49PE4DlEUodvtot1uQ0qJkZERs8Lkg53mJefBRX3YCR+uYeB+FvEVusdx08/w89UL2LQ8qJLB5xVfte1n+BXxCDKCuF4wCC/1tSMQGS0NomxzY41kV1EkC3fG8X6RDCHHkutg4nX5jCL+1+v1AGj5WdTnIiAc+nhLPz7HcU5ylvQYeoa+k0whxdmnZ3Deqpx63H7oa3mdguOG2sbLDYIAt27dwv/0//yfMD8/DwV9f3NzEzMzM/jBD36AS5cuYWhoCE8//TRefvllrK2t4ZVXXsH58+dx5MgR45Tj/XdlIBmlPkfNoHOFynFXO32yrgg4nwZgjY0Pd279RfPffcedez590G2Txfv7dgKIu9A0IYB954DutkKpJrB8XSHpFXbeW5+UEiIMEQkBDDYUu4J+/NQH3CHWz+DrVw7xHuMYKmgX0STnGe68Il1aCIHJyUlsb2+j1Wp528DbJ4XE5OSkVy776nDv82u0gEfRZFxm8XoH0Rn6wd+6YetrkKvo8AnEkVHk1fUpkm59RUySGzMu06aVWrceX5+IKIqAPLOf+MQncOHCBWxtbWWhN0GAz33uc3j77bfx9ttvG6bWbrcxPDyMffv2FdbZz+AAUCi4fIKSl+MSk0/JKDJsuIBxjRJu6JHXP45jayWEv/f000+b93u9HtbX1y3jktrGV7QJyBisVqt499138eKLL2J9fR2NRsO0kZQKCpMkBVgpbVBKKVGr1fDBBx9gfX0dExMTxvtbhA+Ox34GXNFverfX66FSqVhhYr73XLzzfrirSu5Y8SgGHtXAx9UXQl1k8Pj6zqMd+inMPuDMjfexHxPn313PH2+zD3zjNahC4iuL6Nt1SvmUB9+1onnkA+qvS4dCCOO4qVarqNVqGBkZwczMDCYnJ40gofDO1dVV3Lt3D+fPn/cqo2EYYn19Ha+//jpefullrKyu4MCBA1hZWcHExATGxsawvb2Ne/fumXdnZmawsrKCWq2WcyT6okl4+10F0tdvl4e5NMPpaCelgd8n51u320Wv10Oj0cDa2hqq1WqOB/P32+224Wv8ej/6L1IyeZ9obhfxFRdPPlxyXPmUB9dAcMvqp0QVAcc9yTfOk3xztWiOlkolY2TQc7tRdOjZKIoQyhBA8ep10TWSOUX6hY8muLFFny4eOAxC96TUUpmurHXnj/s+kCnYtPpD5uUgPJ7rZDyk+f79+7h48SLm5+dx4MABfOYzn8HQ0FCOP3H8JUkClSSAE51j0TCyVXZXZ+ErnRyntPXpD//wD3F//r4pq9lsIgxDnDhxAnEco9VqodPp4K233sLGxgaCIECr1cJ3v/tdPP3000ZHGBQGjULwQT9HwCC/fePFcd1vju0kY/h7g+o0bhnG6VBYExB1gQ++myDq6AeXrwMHHgeaSwrzl/u82Ke9fzcwWNmuXePTZTjfLZJT7n36TQ5XdzsCpyNyoPn0VLine truncated
},
"metadata": {}
}
],
"source": [
"import supervision as sv\n",
"from inference.models.utils import get_roboflow_model\n",
"\n",
"# Load a pre trained yolov8 nano model from Roboflow Inference.\n",
"model = get_roboflow_model('yolov8n-640')\n",
"\n",
"# Create a video info object from the video path.\n",
"video_info = sv.VideoInfo.from_video_path(path_to_video)\n",
"\n",
"# Create a label annotator for labeling detections with our tracker_id.\n",
"label = sv.LabelAnnotator()\n",
"\n",
"# Create a ByteTrack object to track detections.\n",
"byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
"\n",
"# Create a frame generator from video path for iteration of frames.\n",
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
"\n",
"# Grab a frame from the frame_generator.\n",
"frame = next(frame_generator)\n",
"\n",
"# Run inference on the frame by passing it to our model.\n",
"result = model.infer(frame)[0]\n",
"\n",
"# Convert model results to a supervision detection object.\n",
"detections = sv.Detections.from_inference(result)\n",
"\n",
"# Update detections with tracker ids fro byte_tracker.\n",
"tracked_detections = byte_tracker.update_with_detections(detections)\n",
"\n",
"# Create labels with tracker_id for label annotator.\n",
"labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
"\n",
"# Apply label annotator to frame.\n",
"annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
"\n",
"# Display the frame.\n",
"sv.plot_image(annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wR2nrNUiVOGq"
},
"source": [
"## Tracking Objects in a Video\n",
"\n",
"Finally, we'll use a utility called `VideoSink` to save the annotated frames to a video. Let's dive in to the code."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"metadata": {
"id": "b4lOhjcfVYAl"
},
"outputs": [],
"source": [
"import supervision as sv\n",
"from supervision.assets import download_assets, VideoAssets\n",
"from inference.models.utils import get_roboflow_model\n",
"from tqdm import tqdm\n",
"\n",
"# Load a pre trained yolov8 nano model from Roboflow Inference.\n",
"model = get_roboflow_model('yolov8n-640')\n",
"\n",
"if __name__ == '__main__':\n",
"# Create a video info object from the video path.\n",
"video_info = sv.VideoInfo.from_video_path(path_to_video)\n",
"\n",
" # Install our video from supervision assets.\n",
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
"# Create a label annotator for labeling detections with our tracker_id.\n",
"label = sv.LabelAnnotator()\n",
"\n",
" # Load a pre trained yolov8 nano model from Roboflow Inference.\n",
" model = get_roboflow_model('yolov8n-640')\n",
"# Create a ByteTrack object to track detections.\n",
"byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
"\n",
" # Create a video info object from the video path.\n",
" video_info = sv.VideoInfo.from_video_path(PATH_TO_VIDEO)\n",
"# Create a frame generator from video path for iteration of frames.\n",
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
"\n",
" # Create a label annotator for labeling detections with our tracker_id.\n",
" label = sv.LabelAnnotator()\n",
"# Create a video sink context manager to save resulting video.\n",
"with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
"\n",
" # Create a ByteTrack object to track detections.\n",
" byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
" # Iterate through frames yielded from the frame_generator.\n",
" for frame in tqdm(frame_generator, total=video_info.total_frames):\n",
"\n",
" # Create a frame generator from video path for iteration of frames.\n",
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
" # Run inference on the frame by passing it to our model.\n",
" result = model.infer(frame)[0]\n",
"\n",
" # Create a video sink context manager to save resulting video.\n",
" with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
" # Convert model results to a supervision detection object.\n",
" detections = sv.Detections.from_inference(result)\n",
"\n",
" # Iterate over frames of the video.\n",
" for frame in frame_generator:\n",
" # Update detections with tracker ids fro byte_tracker.\n",
" tracked_detections = byte_tracker.update_with_detections(detections)\n",
"\n",
" # Run inference on the frame by passing it to our model.\n",
" result = model.infer(frame)[0]\n",
" # Create labels with tracker_id for label annotator.\n",
" labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
"\n",
" # Convert model results to a supervision detection object.\n",
" detections = sv.Detections.from_inference(result)\n",
" # Apply label annotator to frame.\n",
" annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
"\n",
" # Update detections with tracker ids fro byte_tracker.\n",
" tracked_detections = byte_tracker.update_with_detections(detections)\n",
"\n",
" # Create labels with tracker_id for label annotator.\n",
" labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
"\n",
" # Apply label annotator to frame.\n",
" annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
"\n",
" # Save the annotated frame to an output video.\n",
" sink.write_frame(frame=annotated_frame)"
" # Save the annotated frame to an output video.\n",
" sink.write_frame(frame=annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"id": "0J10FczGULVe"
},
"source": [
"# View the Video\n",
"Let's take a look at our resulting video. It will also be created in your current directory with the name `output.mp4` Notice how even with a little flicker, we can see the `tracker_id` on the people walking in the video. With trackers under your belt, there are now a wide variety of use cases you can solve for! Happy building!\n",
"\n",
"Let's take a look at our resulting video. It will be created in your current directory with the name `output.mp4` Notice how even with a little flicker, we can see the `tracker_id` on the people walking in the video. With trackers under your belt, there are now a wide variety of use cases you can solve for! Happy building! "
"<div style=\"display: flex; justify-content: center;\">\n",
" <video controls width=\"1000\">\n",
" <source\n",
" src=\"https://media.roboflow.com/supervision/cookbooks/object-tracking.mp4\"\n",
" type=\"video/mp4\"\n",
" >\n",
" </video>\n",
"</div>"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "V100",
"provenance": []
},
"kernelspec": {
"display_name": "venv",
"language": "python",
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
@@ -144,9 +311,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 0
}
File diff suppressed because it is too large. Load diff
+376 -253
View File
File diff suppressed because it is too large. Load diff
+745
View File
@@ -0,0 +1,745 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "RFjrV07Spmm2"
},
"source": [
"# Serialise Detections to a CSV File\n",
"\n",
"---\n",
"\n",
"[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/detections-to-jsonsink.ipynb)\n",
"\n",
"This cookbook introduce [sv.CSVSink](https://supervision.roboflow.com/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) tool designed to write captured object detection data to file from video streams/file"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aFQHoaDLp8R3"
},
"source": [
"Click the `Open in Colab` button to run the cookbook on Google Colab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hzMGlRKlel4p"
},
"outputs": [],
"source": [
"!pip install -q inference requests tqdm supervision==0.21.0"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "6vRfXc_Je5Ee"
},
"outputs": [],
"source": [
"import csv\n",
"from typing import List\n",
"from collections import defaultdict\n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"import supervision as sv\n",
"from supervision.assets import download_assets, VideoAssets\n",
"from inference import InferencePipeline\n",
"from inference.core.interfaces.camera.entities import VideoFrame\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d4s3Ndsowl81"
},
"source": [
"The parameters defined below are:\n",
"* `SOURCE_VIDEO_PATH` - the path to the input video\n",
"* `CONFIDENCE_THRESHOLD` - do not include detections below this confidence level\n",
"* `IOU_THRESHOLD` - discard detections that overlap with others by more than this [IOU](https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy/) ratio\n",
"* `FILE_NAME` - write the json output to this file\n",
"* `INFERENCE_MODEL` - model id. This cookbook uses a [model alias](https://inference.roboflow.com/quickstart/aliases/), but it can also be a [fine-tuned model](https://inference.roboflow.com/quickstart/explore_models/) or a model from the [Universe](https://inference.roboflow.com/quickstart/load_from_universe/)."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"id": "urRjZjh2f30v"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"people-walking.mp4 asset download complete. \n",
"\n"
]
}
],
"source": [
"SOURCE_VIDEO_PATH = download_assets(VideoAssets.PEOPLE_WALKING)\n",
"CONFIDENCE_THRESHOLD = 0.3\n",
"IOU_THRESHOLD = 0.7\n",
"FILE_NAME = \"detections.csv\"\n",
"INFERENCE_MODEL = \"yolov8n-640\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XQnVdIEcjVhy"
},
"source": [
"As a result of executing the above `download_assets(VideoAssets.PEOPLE_WALKING)` , you will download a video file and save it at the `SOURCE_VIDEO_PATH`. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
"\n",
"<video controls width=\"1280\">\n",
" <source src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\" type=\"video/mp4\">\n",
"</video>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nf3sxLWfwrGL"
},
"source": [
"## Read single frame from video\n",
"\n",
"The [`get_video_frames_generator`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.get_video_frames_generator) enables us to easily iterate over video frames. Let's create a video generator for our sample input file and display its first frame on the screen.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "BG8Dqn1WwsPa",
"outputId": "46d4ada1-4e61-4b47-fca4-352b013cd8d1"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9aYxm13UeCj/7DO9U89DzzG52N4emJJKiSM2D6UGwEg8IktzERoxc4PvhAAECBMiPAPkd4Lt/vuT+8UWundgZbMWyJVmWYFoWZYqSSIrsJrubYpPdzZ6qu6qrqmt+x3PO/n7ss9ZZe5993qpm7ocPBmo3GlX1vufsYe2117zXUlprjd2223bbbtttu2237bbdttt2227bbbvt72gL/v89gd2223bbbtttu2237bbdttt2227bbbvtf6XtKra7bbfttt2223bbbtttu2237bbdttv+TrddxXa37bbdttt2227bbbttt+223bbbdtvf6bar2O623bbbdttu2227bbfttt2223bbbvs73XYV292223bbbtttu2237bbdttt2227bbX+n265iu9t2227bbbttt+223bbbdttu22277e9021Vsd9tu2227bbfttt2223bbbtttu223/Z1uu4rtbtttu2237bbdttt2227bbbttt+22v9NtV7Hdbbttt+223bbbdttu2227bbfttt32d7pFO33w3/27fzf0e6UU/x/WtNbQWlvPK6UQBAGiKEIQFLp2VZ/UBwBkWYYkSZCmaalf6gMAgiCw+nHHj6IIURSVnvHNXynFc8iyjPv4H//jf+DixYt45plncOTIEUxNTaHVavHYNGetNTY3N5EkCTY2NjA3N4eDBw/i4x//OKKo2JIwDC14yLVnWcZ/07Pbzd3th+BHv8t35D5J2NHnNAf3/SzLMBgMkCQJf5dlGS5cuIC//uu/Rr1ex969e/GFL3wBjUYDt27dwttvv4379+/z/lN/cRzjl37pl/Cxj30MaZpa+ypxgNYfhiH++3//73j//fdx8OBBjI+Po9PpIIoi1Ot1JEmCwWCATqeDXq+HlZUVRFGEyclJ/Mt/+S8xNTXFc1ZKMUyrYCn3IY5jhGFozWm7vdBaI01TAEAURQjDELVazfsO4TmNmaYpkiRBkiQ81zAMuV/aM9q/IAh4PXK/lFLch3tGfPOXOCy/j+MYURR51+/7jD6XMIjj2JpfVT/yp9Yag8EAWZZZZ2cn77ut0WjYa4UGFMz/IS3NzF7EUTx0r32fZbqgX/V6ffictb8fel8phWazibv37uIP/8sfoj/oQ2caURwhCg1sRkZH8Pf/3t/H9PR09Tjis7gW40//55/iL//yLxHHMV588UW88MILmJycxH/4j/8B165ew6lTp/Av/sW/QK1es+hhoHZoN1WAzjTCMEQcx0MfdedHLdPm3MZR7P2+qmVZhrm7c1hfW8fS0hJu3b6Fffv2YXRkFFEclfZTZ+b8zczO4JFHHrHGytIMtXrNotk600PnIftP0gQAGA/c99yzoZRCGBiao2HG6ff6aLVaNv9zUJJwqfQZgF6vhyAI+CxCofTssEZ0KYxCKCjeF996fX9nWYZkkBg66DlKVfQ1UAW9S9MUmc4QRzFChJVzHbYvg8GAZQKal4t7kv5Tk7w+TVM0Gg1+3uXlcv5VcyC65pNjquYv++v3+1Yf/F2AEnxLPB4aaZJCQzNtc3mHi5MWbYaGgkK/34eGNjxOhQhyf4qXJkJDK/vzQWLkiSiMENdiLz7SPLIsw1vn38L/9Xv/F7Isw4EDB/Drv/7r+O3f+m2Mj43jvSvv4b/80X/B4UOHMTY2hnq9jg8//BCnz5zGxMQEsjTjfcq0vbdaG3hAgemUi9sKDv/wtCzLTD/0TqDoBS9cpKxD55zeJbzXWiNJE+hMyHJK7BOKvfKti9+pWAfjRE7PgjBgeKRZWnqfYeIgmZy/hAc0oAJDz9w+JH1z5yz/ztIMmTYyaRAEhufl9KuKBrlypEuvZOMzmPO1Kv1A9uOj11Xvuv0kacJ4UjVv2sssy3jvAcNLkoGRF9Ms5e96/R5++PIPcfHiRea7WWbgJt9353H69Gn8w3/4D1k+pzNiyeBBaOi+Ky+KbklnYDjTd4p+5PREAXEUIwgDi9b87v/rd4fCjdqOFdvt2k6Yl1Q85XfbCd6+Pl3FzjqUvgPmKLLyM59SK5HSp1i7PxuNBj73uc/h4sWL+OlPf4rz588jiiKMjo6i2WwiiiL0ej0kSYJ2u41Op4MjR47g6aefxsTEhFeJGob8BCcJP/d7lwG6e1Qiyg7cpHJb1a/7HiF9EAQW0z916hR6vR5GR0cxNTWFIAhw9+5dvP7669ja2kKj0cDU1BTCMMT9+/etcX0EwoVPo9HA8vIy2u02RkZGmPGEYYjBYMD7LPccACYnJ5FlGRYWFjAzM2ONsZ0QIRXH7ZSqKoUxSRKGl1TsfXgtYZGmqRHgssxSaAFYSq0Phj6msB2hrWoSB6vOzDBlj/67a5DPuL+TACOVfGkQ8eH2do0MCpXN7UIIIZL5PGwjYaFK6HWflT+tPnJhI8sy1Go1xDVjaJBMTSmFLM1w89ZNTE5O+o1mQoDIsgxhGqLX7xkGlyS4fec2ntfPI67FOH78OK5fu44rV67g9Tdex5e++CV+VmttBDZlCzdeYTYzzDMIAqjAzNG7fpqbFkKjBqAK4agkNGl/X0DBRLM0Nxrlwne/10c/7hd0VVXgcz52aX5D2k5wcZhxxzd/V4Am2CuUlVr5jKdDQBnlNI5j0w92OBdBZ9IsRaQiM744J1Vrl/gRqMAoVFnKNIG+9ykKUggvnYuH0ciddYRhyAY3mof7HK2pSuAmQ6qkbTsRaH00lGjkdkqt/I7osmsU3s5QJ58N4sCisb5zoAJlCcWW8qRyWGbG8aCVvb7tcEtrjSiMeD/ctShlcGyQDBAGIcsUG5sbqMU1DAYDXLhwAZOTk/jiF76IH//kx3j55ZfRbrehM42nn3kaX/2Vr2JmegaZzixc8s0vCAOzVkGDpOIo4e82VogCBR3qSkXC/bvqjAM5fctprFKKYzCrlFRf8z27Ezyj51zeK2FReh6qtBYVKms99Ix8dqfzyFKjoGXIoIOCfv2vyDjbNVdfIbqZpmmJd9Bz2zWGawWelMaX8kiWWjyU8CLLMsRxjE9/+tO4efMmVldXvQp4aS5Q2L9vP5qNJgbJoFJ+dPUXGtvab2UcAVmW2XiSP5IhN3JsR6S2af+PKbY7aT5C4SoQvgPlElMSalmgG6KASMVPAp/eC8PQUmp9B6CKyFmWnizDmTNn8MILL+Cll14CYASEdruNMAxx4MAB3Llzh61JgLHIHj9+HJ1OBxMTE5a3Yjvkl2vxrd+FWRXBdPuTypFPGXKfc8chBY2eSRLjhajVanjiiSewtbWFwWCAwWCAkZERHD16FFEUYXx8HGNjY1haWsLy8jIzMIKtuy9ScabPr169itXVVWs+tVoN9XodnU6H4UQCQr1ex8zMDAaDAT744AM88cQTvG4J32FKGjF8gpErjPjekd9nWYZGo1FSaocxFhI0SPBycVrumVReJExdGLpzlvOuwn/ZT9Vct2vy/A4TFOVP+k9wkHMY1uQY7j5V7aF81zcn11q93djW2pFbwKHZo7pd851b+kkW6larxUakJEnQ6/agoRGqEEma4NbNW3jsscfQbDTtOeXWbcAom2maYnljGUtLS1CBYdSD/gBplkJBodloMvzff/99vPDCCyX8VWpnnluOQhgktjLK+qvAA1JgxXwZD8LyWFW8RMPQp36/j263izAIMTExgV6vV60UER/WhUdKzlVnRnhnL0w+jisw+/qVz/j4mLsmOQYpsQx/IRz64OA9q9r8tzzOOzzT8lxlqdnLMAwRqACpTq3nfOuRf2dphoE2igoJhrSm7RR1hq82/WRBVhKSdnJWgyBg3iX5rI+2Vu1VGIZseNS6iExxn5P0SMo2gJERAL/hbSdKoc/oDaCMu3JOJIzrQsHfam+hHtYL+OTnLEszqEAxL+H9JaVEG0UuQNnwWSnrKc98UMCH8IG8X5k2ML7x4Q38H//v/wP3Fu4BMB6kjY0N3L5zG3/913+Nd955B81mE888/Qxe/uHLWFtdw6uvvoozZ87g8OHDxruti4gTlw8HKjCKudL8HK1VrsNrgBGLYiUs9y4is/nDw5w5opW0x3JOPkWDnvGN4eONhAfuuizjCQKjSDpn2ze2l/5JAyEZDPKfw+Tu0niBQqQig3NiD13a8FHkFHpf8rLt+iGvtpwjNZ884TYF4+HNVGYU9ornpDyUaRPxwp/n55zOUZIlgAKmZ6axtbWFdqftNV66ODAzM1OSFd25+/DN3W+ScweDgVHYla2YkyL8v2qE+H9csd0OmV3h0iVwLvEDbO9s1QZUEUn63KfY+pTaqjUppbwbS20wGCAIAnzmM5/B+fPnsbGxwZsYhiG+/OUv48KFC7hw4QIzgW63i9HR0ZKnUI5ZSfzzJhmXC2cfTH0KgjuOSySrlGKfgiAFALL6A2CPNYW4BUGARqOBZ555hvdjMBhgbW3NWjutTxonaCwK1W40Grh8+TK+//3vY21tDWNjYzxHEioajQYLzzSner0OpUz45vvvv4+1tTVMT0+zF1Wuz7dWqZS5+zLsbym01Ov1Iuyv4j13z6RC586R9oAEACs01BFwCJ99IcQuvvmEYhp3J2eoqsl5SsVenrVhyq5cn2xVhNEH54/K6Ijpk4fL/c5Hqyy4QrGA5F5X8DUShKy15YJjp92BUgr1sI6R1gjGx8cxd2cOMzMzCIIA/UEfYc0YdFZWVnDn9h2cOXOmJIwCZj/X1tbw+uuv4+WXX8bKygr27duHBw8eYHp6GlOTU9jc2sTt27f53bk7c1heXkar1UIYmHOZBRkCZX6W4EbTp3OkgoLJ5esyP/K9zoTnILine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH)\n",
"frame = next(generator)\n",
"\n",
"sv.plot_image(frame, (12, 12))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2YyE30JOS4Sf"
},
"source": [
"## Initialize ByteTrack\n",
"\n",
"[ByteTrack](https://supervision.roboflow.com/latest/trackers/#bytetrack) is a multi-object tracking algorithm used by Supervision to track and link detected objects across multiple frames, providing consistent IDs for each object.Initialize the object."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "Jpw_TzAsm2oL"
},
"outputs": [],
"source": [
"byte_track = sv.ByteTrack(minimum_consecutive_frames=3)\n",
"byte_track.reset()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IjYwj2vbTLBd"
},
"source": [
"## Initialize CSVSink\n",
"\n",
"To save detections to a `CSV` file, open our [`sv.CSVSink`](https://supervision.roboflow.com/latest/how_to/save_detections/#save-detections-as-csv) and then pass the [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) object resulting from the inference to it.\n",
"\n",
"Note that empty detections will be skipped."
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"id": "-yvv6N1uTKSl"
},
"outputs": [],
"source": [
"csv_sink = sv.CSVSink(FILE_NAME)\n",
"csv_sink.open()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-YvmJW5yud1G"
},
"source": [
"## Process video and save detections to csv file\n",
"\n",
"The [`InferencePipeline`](https://inference.roboflow.com/using_inference/inference_pipeline/) interface is made for streaming and is likely the best route to go for real time use cases. It is an asynchronous interface that can consume many different video sources including local devices (like webcams), RTSP video streams, video files, etc. With this interface, you define the source of a video stream and sinks.\n",
"\n",
"All the operations we plan to perform for each frame of our video - detection, tracking, annotation, and write to csv are encapsulated in a function named `callback`."
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"id": "eRQiVkblapCk"
},
"outputs": [],
"source": [
"def callback(predictions: dict, frame: VideoFrame) -> np.ndarray:\n",
" detections = sv.Detections.from_inference(predictions)\n",
"\n",
" # Only keep person detections\n",
" detections = detections[detections.class_id == 0]\n",
" detections.data[\"class_name\"] = np.array([\"person\" for _ in range(len(detections))])\n",
"\n",
" detections = byte_track.update_with_detections(detections)\n",
" csv_sink.append(detections, custom_data={'frame_number': frame.frame_id})"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3GRH_QdHgGiC",
"outputId": "2baa2f16-29e7-49a1-b2ce-5dc26397c189"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n",
"UserWarning: Specified provider 'OpenVINOExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n"
]
}
],
"source": [
"pipeline = InferencePipeline.init(\n",
" model_id=INFERENCE_MODEL,\n",
" video_reference=SOURCE_VIDEO_PATH,\n",
" on_prediction=callback,\n",
" iou_threshold=IOU_THRESHOLD,\n",
" confidence=CONFIDENCE_THRESHOLD,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"id": "bYpA9VxBsOvS"
},
"outputs": [],
"source": [
"pipeline.start()\n",
"pipeline.join()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"id": "RrewOF02AYta"
},
"outputs": [],
"source": [
"csv_sink.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "u7Akx7aUsh75"
},
"source": [
"## Visualizate results of detections csv data with Pandas\n",
"\n",
"Let's take a look at our resulting data with by using Pandas.\n",
"\n",
"It will also be created in your current directory with the name detections.csv as well."
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 424
},
"id": "Bu7AZ3QHiAqb",
"outputId": "a0da3923-b9b7-4737-c8df-a063d2d8453d"
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>x_min</th>\n",
" <th>y_min</th>\n",
" <th>x_max</th>\n",
" <th>y_max</th>\n",
" <th>class_id</th>\n",
" <th>confidence</th>\n",
" <th>tracker_id</th>\n",
" <th>class_name</th>\n",
" <th>frame_number</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1142.0</td>\n",
" <td>950.0</td>\n",
" <td>1245.0</td>\n",
" <td>1080.0</td>\n",
" <td>0</td>\n",
" <td>0.767850</td>\n",
" <td>185</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>750.0</td>\n",
" <td>450.0</td>\n",
" <td>823.0</td>\n",
" <td>620.0</td>\n",
" <td>0</td>\n",
" <td>0.748268</td>\n",
" <td>168</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1419.0</td>\n",
" <td>702.0</td>\n",
" <td>1526.0</td>\n",
" <td>887.0</td>\n",
" <td>0</td>\n",
" <td>0.550286</td>\n",
" <td>179</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1674.0</td>\n",
" <td>12.0</td>\n",
" <td>1727.0</td>\n",
" <td>141.0</td>\n",
" <td>0</td>\n",
" <td>0.546864</td>\n",
" <td>174</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1616.0</td>\n",
" <td>18.0</td>\n",
" <td>1655.0</td>\n",
" <td>127.0</td>\n",
" <td>0</td>\n",
" <td>0.520566</td>\n",
" <td>181</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8757</th>\n",
" <td>394.0</td>\n",
" <td>471.0</td>\n",
" <td>460.0</td>\n",
" <td>633.0</td>\n",
" <td>0</td>\n",
" <td>0.466400</td>\n",
" <td>244</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8758</th>\n",
" <td>115.0</td>\n",
" <td>256.0</td>\n",
" <td>199.0</td>\n",
" <td>399.0</td>\n",
" <td>0</td>\n",
" <td>0.431374</td>\n",
" <td>223</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8759</th>\n",
" <td>77.0</td>\n",
" <td>700.0</td>\n",
" <td>169.0</td>\n",
" <td>892.0</td>\n",
" <td>0</td>\n",
" <td>0.416318</td>\n",
" <td>240</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8760</th>\n",
" <td>1010.0</td>\n",
" <td>59.0</td>\n",
" <td>1055.0</td>\n",
" <td>174.0</td>\n",
" <td>0</td>\n",
" <td>0.403422</td>\n",
" <td>246</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8761</th>\n",
" <td>1758.0</td>\n",
" <td>0.0</td>\n",
" <td>1809.0</td>\n",
" <td>96.0</td>\n",
" <td>0</td>\n",
" <td>0.313793</td>\n",
" <td>236</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>8762 rows \u00d7 9 columns</p>\n",
"</div>"
],
"text/plain": [
" x_min y_min x_max y_max class_id confidence tracker_id \\\n",
"0 1142.0 950.0 1245.0 1080.0 0 0.767850 185 \n",
"1 750.0 450.0 823.0 620.0 0 0.748268 168 \n",
"2 1419.0 702.0 1526.0 887.0 0 0.550286 179 \n",
"3 1674.0 12.0 1727.0 141.0 0 0.546864 174 \n",
"4 1616.0 18.0 1655.0 127.0 0 0.520566 181 \n",
"... ... ... ... ... ... ... ... \n",
"8757 394.0 471.0 460.0 633.0 0 0.466400 244 \n",
"8758 115.0 256.0 199.0 399.0 0 0.431374 223 \n",
"8759 77.0 700.0 169.0 892.0 0 0.416318 240 \n",
"8760 1010.0 59.0 1055.0 174.0 0 0.403422 246 \n",
"8761 1758.0 0.0 1809.0 96.0 0 0.313793 236 \n",
"\n",
" class_name frame_number \n",
"0 person 1 \n",
"1 person 1 \n",
"2 person 1 \n",
"3 person 1 \n",
"4 person 1 \n",
"... ... ... \n",
"8757 person 341 \n",
"8758 person 341 \n",
"8759 person 341 \n",
"8760 person 341 \n",
"8761 person 341 \n",
"\n",
"[8762 rows x 9 columns]"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.read_csv(FILE_NAME)\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pmGzIUqRkpqI"
},
"source": [
"## Convert CSV data to sv.Detections"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"id": "aa24sW8FI-xV"
},
"outputs": [],
"source": [
"def csv_to_detections(csv_file: str) -> List[sv.Detections]:\n",
" rows_by_frame_number = defaultdict(list)\n",
" with open(csv_file, 'r') as f:\n",
" reader = csv.DictReader(f)\n",
" for row in reader:\n",
" frame_number = int(row[\"frame_number\"])\n",
" rows_by_frame_number[frame_number].append(row)\n",
"\n",
" detections_list = []\n",
" for frame_number, rows in rows_by_frame_number.items():\n",
" xyxy = []\n",
" class_id = []\n",
" confidence = []\n",
" tracker_id = []\n",
" custom_data = defaultdict(list)\n",
"\n",
" for row in rows:\n",
" xyxy.append([row[key] for key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\"]])\n",
" class_id.append(row[\"class_id\"])\n",
" confidence.append(row[\"confidence\"])\n",
" tracker_id.append(row[\"tracker_id\"])\n",
"\n",
" for custom_key in row.keys():\n",
" if custom_key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\", \"class_id\", \"confidence\", \"tracker_id\"]:\n",
" continue\n",
" custom_data[custom_key].append(row[custom_key])\n",
"\n",
" if all([val == \"\" for val in class_id]):\n",
" class_id = None\n",
" if all([val == \"\" for val in confidence]):\n",
" confidence = None\n",
" if all([val == \"\" for val in tracker_id]):\n",
" tracker_id = None\n",
"\n",
" detections_list.append(\n",
" sv.Detections(\n",
" xyxy=np.array(xyxy, dtype=np.float32),\n",
" class_id=np.array(class_id, dtype=int),\n",
" confidence=np.array(confidence, dtype=np.float32),\n",
" tracker_id=np.array(tracker_id, dtype=int),\n",
" data=dict(custom_data)\n",
" )\n",
" )\n",
" \n",
" return detections_list"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "EREtElrA0u49",
"outputId": "c83a5658-fb28-4734-a09c-fd94a9e1b45a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detections: 341\n",
"Detections(xyxy=array([[1142., 950., 1245., 1080.],\n",
" [ 750., 450., 823., 620.],\n",
" [1419., 702., 1526., 887.],\n",
" [1674., 12., 1727., 141.],\n",
" [1616., 18., 1655., 127.],\n",
" [1388., 2., 1437., 184.]], dtype=float32), mask=None, confidence=array([0.7678498 , 0.7482683 , 0.5502863 , 0.5468636 , 0.5205659 ,\n",
" 0.31684005], dtype=float32), class_id=array([0, 0, 0, 0, 0, 0]), tracker_id=array([185, 168, 179, 174, 181, 28]), data={'class_name': ['person', 'person', 'person', 'person', 'person', 'person'], 'frame_number': ['1', '1', '1', '1', '1', '1']})\n"
]
}
],
"source": [
"detections_list = csv_to_detections(FILE_NAME)\n",
"detections_list\n",
"\n",
"print(f\"Detections: {len(detections_list)}\")\n",
"print(detections_list[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2pyVFVWI096b"
},
"source": [
"###\u00a0Annotate First Frame\n",
"\n",
"Visualize the first frame of a video alongside the initial detections obtained by parsing CSV data into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) objects. The annotated image will show the original video frame, marked with the first bounding box detected from the parsed data, providing a visual representation of the identified object(s) in the scene."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qXtulNDT1ChB"
},
"source": [
"### Get back `sv.Detections`"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"id": "YRYFmKxR05xG"
},
"outputs": [],
"source": [
"FRAME_NUMBER = 100\n",
"\n",
"detections = detections_list[FRAME_NUMBER]\n",
"frame_number = detections.data[\"frame_number\"][0]\n",
"\n",
"\n",
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH, start=FRAME_NUMBER)\n",
"frame = next(generator)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oIK2K9zM1Kn9"
},
"source": [
"###\u00a0Annotate Image with Detections\n",
"\n",
"Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) classes."
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "WBv8apPz1IiP",
"outputId": "ad32b078-12d5-4457-e16d-c7a428ab858a"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9+ZMdyXUfjn6yqu7at/cFQHdj34EBZ+MsFEVSlERKFL/62pRk+6uIJ4fseBEO23+Hv/HCv7wIR7wX4Rdhh/yVLGsL2xRJieRoZkjOwuHsGGAGwGBpAI1Gb+j19t2rKt8PWSfrZFbW7QZlSeR7fYbgvV23KiuXk2c/J4WUUmIf9mEf9mEf9mEf9mEf9mEf9mEf9uHnFLx/6A7swz7swz7swz7swz7swz7swz7swz78bWBfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+rmFfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+riHY643/7t/9O0gpM9ellOj1eoiiCADg+77+zfd9eJ4HKSU8T+nQQgj9z/M8xHGMOI71Nd/39W8AEMexfo8QwugDfY/jWL/D931EUWTcT7/VajUUi0Wj//R+aoveS9/t/trP8X7w8XGwnw/DEJ1OB1EUodvt6ja63a6ev16vh2KxiOHhYfi+r8dD7fH30hjDMESr1UKz2cTOzo5+l5QSlUoFxWIRQghEUaTbbLfb2NjYwOzsLL75zW9idHQUL774IsrlMgCgUC5gbHwMvufr90ZRBE96+nscx4iiCFEUQQiBIAjg+74eC++3/d2eL3uM9lg5SCmd74jjGL1eT8+5C28I1+r1OlZXV3HgwAFUq9VMv/hz6+vrWF1dxalTpzJ45FoPwrsgCDL4S/3t9XoaFzzPQ6FQ0GvE59T1HgB6nqltwjEppbEPXO/2fR8ff/wxbt68iV7cw5EjR/Dc888BMSCkhfcCiEWs583zPPS6Pbz1k7dw6+YtlCtlBEGAleUVXL9+HcIT6LQ7OHnyJP7V//1foVQq6T7TOKmdvD3D1zOKIpTLZT1WPiYXbvC54nMRBCbJ4/hFuMzxhj6JRrj6uhegvna7XRSLRQRBgDAMAQCbPziF5tXpx27z7wqCQ+sY/I13IQQMvKG5JNrHaX0/sPe853lot9soFAooFAoG/d0NOM0Pw9C5D/l9dh84EN8iWsef5XhFY45kBBlLQJr7l3gcAecpYRjiypUruHHjBoIgwKlTpzA+Po4oivDmm2/i9u3bABSd/uIXv4jnnntO8xUAGt+bzSauXbuGH//4x1hfX0cYhhpf+RiFEBCBwK/8yq9AxAK/8Au/gJmZGeM+414L/yuVisZzTTuFhIR7fYQQQIInvW4PURyhEBQy7zEfMq97IqEBnkDYCyE8kbuu+p2AWgcpICDwcPkh3n77bSw8XEAcxSiVShgYGMCF8xdw4dwFhGEI3/dRKpXwZ3/2Z/jTP/1TY63z9rTrehRF+Kf/9J/iG9/4BqSUuH37Nv7oj/4I7XZb7+tisYhCoYDz58/jn/2zfwav6EHG6bsEBDx4ml4vLy/jj/7oj/DgwQNIKfXeqFarGtejKML4+DharRY++9nP4h//43+MXq+n+yWFhJDZ/hI+BkHQn2YKAB7g+UqOEhDohT1cv3Ydr/7gVVQqFSwtLqHRbGB1dRWLi4vwpIdu2EWlXMH/+f/4PzE7MwsRC4NO23PJebJNX/Qct33U352BjM3xqH1FcprUeEADUHjqkhU4f44RxymOq77RBBhPGX9Vn3gIf6il+93r9SCl1LiaJ2dkppnRQ6IRrvXgPIevmSETS0/NhxAQAbTspeUP4UHAgxTpM/S853kQUG1HcaR5JN8TElK5v3JYnpQSIkp/jKJI8zWSFQivnBBnplm3K6VELGJFXyEheCdiGH/3kwO4HES4BwBhFEKKHL4j03+utj34iBHD80z8tuVW6hsAJ63eje9xnuuSL+2xt1ot/Jf/8l9w48YN+L6PwcFB/It/8S9w4sQJvP322/iv//W/KjrhCXzjG9/AM888o+YHClc8z4Pnp7qa3RcAxnpzeUDjiw80Gg380R/+kdZvzp8/j6989StoN9t47Yev4cqVKxgfHwcA/Mqv/AqOHj2q8YbWiOuP9O/f/tt/23e+CPas2Lo2KAmcAAyF1BayXYI1PRuGoR4EIWCpVEK5XM4I6nY/uPLAgRRmu+9hGGrFwSYuNvK5kI4TKt4vrpjb93GIogiNRgOtVgvFYhGe56FcLutnBwcHjefu3r0L3/cxMjKS6YfrO1cw6XoYhohjxehJ6eQCUafTQaPRgBACJ0+exDvvvIOTJ0/iwIEDCIIAnW4H7VYbpXIJgR+k42TCnb3RyMCQuzEcirlL+XUB/UYbXW8o1gZXTuhf3rxtb2/ruXfNJ0G324WUEgcPHtTCny0s2n0kvHaNjc+XrUhxZS9PMad/3IDAFX3qG82Ti+Hqd/geSoUSarWa2puIIZDtq/QUc5GxYoSe7+HJzzyJe/fuoV6vo1QsQUIiljE6jQ4838O5c+f0XgZS3CCiRWvDDVk2DtgKJWe8XDixwZ5jF9Pha2a348JvG9f2CqQ40PrwPZqjM/xMw14ZM5Clh7TGhAN5+8jVjv09rw82v3H99tMYKCBgrJcLpziulUolXLp0CadOnTIU3l6vh4MHD+L+/fsIwxAjIyOYnp7WijIJpwQDAwM4f/48FhYWUK/X0el0MvhI39utNnYaO5iemsbg4KAh6BLvtfvfd8iJ8trvd9o/e11LZ9sSEN7jr0mhUMDKygo2tzbhez56rR566KERNbC2ugacS++t1+u4evVqhufsFUgR2dra0rLP6dOn8Vu/9Vv4H//jf+h1oTWen5/H6uoqpg9PI4zDTFvtdhtzc3N4++23MTc3h3PnzqFer2uFlZQebvjc2NiA53mGwK4GkhX2OS5SG/x3Pn7P8+AVPAR+AM9XSo/nefCDhKdEMcrlMjzPw/LSMsjoIYRAu9PG/fv3MTsza7RNgqnNtznPIjCU3E6A+oeHIMO9Gc/+7kGidOwR/KGWHksQBOj1egjDUBtM90oTAXOOwjA0DGR5tMu+RusdyxiITEM/GVDU/W75Sv+eGJtIXvxpgWRMwll6R55hbLe2AIUzGR4voZRdaw/bcqhtGKC29rpGxis5rnpCGRNyaAh/r/0s37e2nGyvO9Frl8xMTg++z+/fv4+7d+/q+Xr++ecxMTGBnZ0dXLt2TT87PT2NkydPQngCQgrEkTLcqpf2wWOm7FNbXHci+a/b7WpcKpfLmD08i263i3a7jTt37mheeOzYMUxNTRlGFRq3S7fbK+xZseUv4wMiRLEVCL6ByBMphGmJ9X0fhUJBt+37PpaXlxFFEarVqvEeThxt5OVMnTNuu6+9Xk9bQglc7bqIif1bFEWaqAGppcz1HI1va2sLGxsbOHjwoFaweR/4WJvNJorFouHpygNbyeFKSxwrJCOFt1Qqwfd9zWDK5TLGxsbQ7XY1gt2+fRvVahUTkxPoRT20O23Dq+EJTyM/N2iQoOoHvjmnibXN3ix8zezrnPFxQusiFLytKIoQBAFKpVIuzvBnDx48iPX1dWxtbWmLdhRFKBaLRh+EEBgdHUWhUMgoinpemHWJK6nUP1v4BVKibSu/XOGz+83H1e8d9vjtdwshsLGxARlLlCtljIyOaI9U5nnFSRAjhozUmgZBgIGBAVy8cBHvvf8eAKDZbGKwNogDUwcwPT2NJ598ElGsPF18f7jGZP/GgXul97Jf+VhtWmHPgS3kut7/0ypCfF8TbgL4WwkPP0uQRytdwK/3i8TZC9g8Jk+5zfvb3m+uNjLKqhRamHK1Rd/5/hdCoFwu63Unhu/7PqanpzExMYH19XWMjIxgcHAQnU5HKw9CKKMTRX10u10sLS2h2WwaPJXvIbp2+tRpPPWZpzAwMGAIRgAy9MJeF1v54UauPBBCeVxJsPHE3gUSrowJoYSsfvtN81SY0TXtVlsbj/22r3ks8bs4jrGwsIC5uTmthPfDuzy8JAGy2Wxqj+qlS5dw48YNfPTRR4YBo9ls4u69u5iankIsYwR+oHBfKD76zjvv4Hvf+x52dnZQrVZx/PhxDA8PY2lpCTdu3NCKLeER8ezJyUl0u11jLbnAacsWXC5yjcv3fRRLRQwMDqBaUbJXq91CHMeoDdQACew0dhCGIdqdNvxA8XpIwA98yFjik48/wckTJzE6PKqeYfPFP20c+2no6j8ESGnOJymkpJS6+Ju7nf6GOH7Ndb+hzEhlaZMQkDI2DGMAtMc2hlsWQvI8jYfolBH1uItSyr2p3EBN/8gDmAd5ih19J0eM5p0yVkYwy9pmy2Wcx3JDkL5PoO/YbLrE+yel8nr24z+2/mEogOx+2/nhklVdijPXv6RU0WA/+clPtFFsZmYGL774IiqVCu7cuYO5uTn9/BMXn8D4+DhiGWfajiPzmvG7hJIRHWOWUjk8QqkiR7u9LsJeiMGhQe2c63a72rkXhiEmJibg+UnkbrKupLBzw4TNc3eDPSu2HGE5gvDNkGGIQmB7exvdbhdDQ0OawbiECa7MjYyMZLxstgDEF5yEeztckPpJ16MoQrvdBgDtIXZBP8GMIxQPyTOIjSXc0DNBECAIArRaLR0qCyATIkT3zs7OIggCpxDsUgSFECiVSigWi5BSYnt7Wyu2cRyj3W5rIYvCaHzfR7VahZQSQ0NDmJ2dxf379zE9PY1KpYLt5jYOHTqEUrmEOEqEUZjvpjEGQaAEMLine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"bounding_box_annotator = sv.BoundingBoxAnnotator()\n",
"label_annotator = sv.LabelAnnotator()\n",
"\n",
"annotated_frame = frame.copy()\n",
"annotated_frame = bounding_box_annotator.annotate(scene=annotated_frame, detections=detections)\n",
"annotated_frame = label_annotator.annotate(scene=annotated_frame, detections=detections)\n",
"sv.plot_image(annotated_frame, (12, 12))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YRPW1JHK1VW-"
},
"source": [
"##\u00a0References \ud83d\udcda\n",
"\n",
"* Supervision: https://supervision.roboflow.com\n",
"* sv.Detections: https://supervision.roboflow.com/develop/detection/core/#detections\n",
"* Save Detections to CSV: https://supervision.roboflow.com/develop/how_to/save_detections/#save-detections-as-csv\n",
"* Custom fields: https://supervision.roboflow.com/develop/how_to/save_detections/#custom-fields\n",
"* ByteTrack: https://supervision.roboflow.com/trackers/#supervision.tracker.byte_tracker.core.ByteTrack\n",
"* Inference: https://inference.roboflow.com/\n",
"* Inference Pipeline: https://inference.roboflow.com/using_inference/inference_pipeline/\n",
"* Inference Aliases: https://inference.roboflow.com/quickstart/aliases/\n",
"\n"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"machine_shape": "hm",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
+779
View File
@@ -0,0 +1,779 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "RFjrV07Spmm2"
},
"source": [
"# Serialise Detections to a JSON File\n",
"\n",
"---\n",
"\n",
"[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/detections-to-jsonsink.ipynb)\n",
"\n",
"This cookbook introduce [sv.JSONSink](https://supervision.roboflow.com/develop/detection/tools/save_detections/#supervision.detection.tools.json_sink.JSONSink) tool designed to write captured object detection data to file from video streams/file"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aFQHoaDLp8R3"
},
"source": [
"Click the `Open in Colab` button to run the cookbook on Google Colab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hzMGlRKlel4p"
},
"outputs": [],
"source": [
"!pip install -q inference requests tqdm supervision==0.21.0"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "6vRfXc_Je5Ee"
},
"outputs": [],
"source": [
"import json\n",
"from typing import List\n",
"from collections import defaultdict\n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"import supervision as sv\n",
"from supervision.assets import download_assets, VideoAssets\n",
"from inference import InferencePipeline\n",
"from inference.core.interfaces.camera.entities import VideoFrame\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "95av545ydC1O"
},
"source": [
"The parameters defined below are:\n",
"* `SOURCE_VIDEO_PATH` - the path to the input video\n",
"* `CONFIDENCE_THRESHOLD` - do not include detections below this confidence level\n",
"* `IOU_THRESHOLD` - discard detections that overlap with others by more than this [IOU](https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy/) ratio\n",
"* `FILE_NAME` - write the json output to this file\n",
"* `INFERENCE_MODEL` - model id. This cookbook uses a [model alias](https://inference.roboflow.com/quickstart/aliases/), but it can also be a [fine-tuned model](https://inference.roboflow.com/quickstart/explore_models/) or a model from the [Universe](https://inference.roboflow.com/quickstart/load_from_universe/)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "urRjZjh2f30v"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"people-walking.mp4 asset download complete. \n",
"\n"
]
}
],
"source": [
"SOURCE_VIDEO_PATH = download_assets(VideoAssets.PEOPLE_WALKING)\n",
"CONFIDENCE_THRESHOLD = 0.3\n",
"IOU_THRESHOLD = 0.7\n",
"FILE_NAME = \"detections.json\"\n",
"INFERENCE_MODEL = \"yolov8n-640\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0JjJhzuDAgdH"
},
"source": [
"As a result of executing the above `download_assets(VideoAssets.PEOPLE_WALKING)` , you will download a video file and save it at the `SOURCE_VIDEO_PATH`. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
"\n",
"<video controls width=\"1280\">\n",
" <source src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\" type=\"video/mp4\">\n",
"</video>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HZ20JXd4gYdw"
},
"source": [
"##\u00a0Read single frame from video\n",
"\n",
"The [`get_video_frames_generator`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.get_video_frames_generator) enables us to easily iterate over video frames. Let's create a video generator for our sample input file and display its first frame on the screen.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "kRcm6D9agmor",
"outputId": "20e3bb60-236c-446c-b636-ce7ea3f591d0"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9aYxm13UeCj/7DO9U89DzzG52N4emJJKiSM2D6UGwEg8IktzERoxc4PvhAAECBMiPAPkd4Lt/vuT+8UWundgZbMWyJVmWYFoWZYqSSIrsJrubYpPdzZ6qu6qrqmt+x3PO/n7ss9ZZe5993qpm7ocPBmo3GlX1vufsYe2117zXUlprjd2223bbbtttu2237bbdttt2227bbbvt72gL/v89gd2223bbbtttu2237bbdttt2227bbbvtf6XtKra7bbfttt2223bbbtttu2237bbdttv+TrddxXa37bbdttt2227bbbttt+223bbbdtvf6bar2O623bbbdttu2227bbfttt2223bbbvs73XYV292223bbbtttu2237bbdttt2227bbX+n265iu9t2227bbbttt+223bbbdttu22277e9021Vsd9tu2227bbfttt2223bbbtttu223/Z1uu4rtbtttu2237bbdttt2227bbbttt+22v9NtV7Hdbbttt+223bbbdttu2227bbfttt32d7pFO33w3/27fzf0e6UU/x/WtNbQWlvPK6UQBAGiKEIQFLp2VZ/UBwBkWYYkSZCmaalf6gMAgiCw+nHHj6IIURSVnvHNXynFc8iyjPv4H//jf+DixYt45plncOTIEUxNTaHVavHYNGetNTY3N5EkCTY2NjA3N4eDBw/i4x//OKKo2JIwDC14yLVnWcZ/07Pbzd3th+BHv8t35D5J2NHnNAf3/SzLMBgMkCQJf5dlGS5cuIC//uu/Rr1ex969e/GFL3wBjUYDt27dwttvv4379+/z/lN/cRzjl37pl/Cxj30MaZpa+ypxgNYfhiH++3//73j//fdx8OBBjI+Po9PpIIoi1Ot1JEmCwWCATqeDXq+HlZUVRFGEyclJ/Mt/+S8xNTXFc1ZKMUyrYCn3IY5jhGFozWm7vdBaI01TAEAURQjDELVazfsO4TmNmaYpkiRBkiQ81zAMuV/aM9q/IAh4PXK/lFLch3tGfPOXOCy/j+MYURR51+/7jD6XMIjj2JpfVT/yp9Yag8EAWZZZZ2cn77ut0WjYa4UGFMz/IS3NzF7EUTx0r32fZbqgX/V6ffictb8fel8phWazibv37uIP/8sfoj/oQ2caURwhCg1sRkZH8Pf/3t/H9PR09Tjis7gW40//55/iL//yLxHHMV588UW88MILmJycxH/4j/8B165ew6lTp/Av/sW/QK1es+hhoHZoN1WAzjTCMEQcx0MfdedHLdPm3MZR7P2+qmVZhrm7c1hfW8fS0hJu3b6Fffv2YXRkFFEclfZTZ+b8zczO4JFHHrHGytIMtXrNotk600PnIftP0gQAGA/c99yzoZRCGBiao2HG6ff6aLVaNv9zUJJwqfQZgF6vhyAI+CxCofTssEZ0KYxCKCjeF996fX9nWYZkkBg66DlKVfQ1UAW9S9MUmc4QRzFChJVzHbYvg8GAZQKal4t7kv5Tk7w+TVM0Gg1+3uXlcv5VcyC65pNjquYv++v3+1Yf/F2AEnxLPB4aaZJCQzNtc3mHi5MWbYaGgkK/34eGNjxOhQhyf4qXJkJDK/vzQWLkiSiMENdiLz7SPLIsw1vn38L/9Xv/F7Isw4EDB/Drv/7r+O3f+m2Mj43jvSvv4b/80X/B4UOHMTY2hnq9jg8//BCnz5zGxMQEsjTjfcq0vbdaG3hAgemUi9sKDv/wtCzLTD/0TqDoBS9cpKxD55zeJbzXWiNJE+hMyHJK7BOKvfKti9+pWAfjRE7PgjBgeKRZWnqfYeIgmZy/hAc0oAJDz9w+JH1z5yz/ztIMmTYyaRAEhufl9KuKBrlypEuvZOMzmPO1Kv1A9uOj11Xvuv0kacJ4UjVv2sssy3jvAcNLkoGRF9Ms5e96/R5++PIPcfHiRea7WWbgJt9353H69Gn8w3/4D1k+pzNiyeBBaOi+Ky+KbklnYDjTd4p+5PREAXEUIwgDi9b87v/rd4fCjdqOFdvt2k6Yl1Q85XfbCd6+Pl3FzjqUvgPmKLLyM59SK5HSp1i7PxuNBj73uc/h4sWL+OlPf4rz588jiiKMjo6i2WwiiiL0ej0kSYJ2u41Op4MjR47g6aefxsTEhFeJGob8BCcJP/d7lwG6e1Qiyg7cpHJb1a/7HiF9EAQW0z916hR6vR5GR0cxNTWFIAhw9+5dvP7669ja2kKj0cDU1BTCMMT9+/etcX0EwoVPo9HA8vIy2u02RkZGmPGEYYjBYMD7LPccACYnJ5FlGRYWFjAzM2ONsZ0QIRXH7ZSqKoUxSRKGl1TsfXgtYZGmqRHgssxSaAFYSq0Phj6msB2hrWoSB6vOzDBlj/67a5DPuL+TACOVfGkQ8eH2do0MCpXN7UIIIZL5PGwjYaFK6HWflT+tPnJhI8sy1Go1xDVjaJBMTSmFLM1w89ZNTE5O+o1mQoDIsgxhGqLX7xkGlyS4fec2ntfPI67FOH78OK5fu44rV67g9Tdex5e++CV+VmttBDZlCzdeYTYzzDMIAqjAzNG7fpqbFkKjBqAK4agkNGl/X0DBRLM0Nxrlwne/10c/7hd0VVXgcz52aX5D2k5wcZhxxzd/V4Am2CuUlVr5jKdDQBnlNI5j0w92OBdBZ9IsRaQiM744J1Vrl/gRqMAoVFnKNIG+9ykKUggvnYuH0ciddYRhyAY3mof7HK2pSuAmQ6qkbTsRaH00lGjkdkqt/I7osmsU3s5QJ58N4sCisb5zoAJlCcWW8qRyWGbG8aCVvb7tcEtrjSiMeD/ctShlcGyQDBAGIcsUG5sbqMU1DAYDXLhwAZOTk/jiF76IH//kx3j55ZfRbrehM42nn3kaX/2Vr2JmegaZzixc8s0vCAOzVkGDpOIo4e82VogCBR3qSkXC/bvqjAM5fctprFKKYzCrlFRf8z27Ezyj51zeK2FReh6qtBYVKms99Ix8dqfzyFKjoGXIoIOCfv2vyDjbNVdfIbqZpmmJd9Bz2zWGawWelMaX8kiWWjyU8CLLMsRxjE9/+tO4efMmVldXvQp4aS5Q2L9vP5qNJgbJoFJ+dPUXGtvab2UcAVmW2XiSP5IhN3JsR6S2af+PKbY7aT5C4SoQvgPlElMSalmgG6KASMVPAp/eC8PQUmp9B6CKyFmWnizDmTNn8MILL+Cll14CYASEdruNMAxx4MAB3Llzh61JgLHIHj9+HJ1OBxMTE5a3Yjvkl2vxrd+FWRXBdPuTypFPGXKfc8chBY2eSRLjhajVanjiiSewtbWFwWCAwWCAkZERHD16FFEUYXx8HGNjY1haWsLy8jIzMIKtuy9ScabPr169itXVVWs+tVoN9XodnU6H4UQCQr1ex8zMDAaDAT744AM88cQTvG4J32FKGjF8gpErjPjekd9nWYZGo1FSaocxFhI0SPBycVrumVReJExdGLpzlvOuwn/ZT9Vct2vy/A4TFOVP+k9wkHMY1uQY7j5V7aF81zcn11q93djW2pFbwKHZo7pd851b+kkW6larxUakJEnQ6/agoRGqEEma4NbNW3jsscfQbDTtOeXWbcAom2maYnljGUtLS1CBYdSD/gBplkJBodloMvzff/99vPDCCyX8VWpnnluOQhgktjLK+qvAA1JgxXwZD8LyWFW8RMPQp36/j263izAIMTExgV6vV60UER/WhUdKzlVnRnhnL0w+jisw+/qVz/j4mLsmOQYpsQx/IRz64OA9q9r8tzzOOzzT8lxlqdnLMAwRqACpTq3nfOuRf2dphoE2igoJhrSm7RR1hq82/WRBVhKSdnJWgyBg3iX5rI+2Vu1VGIZseNS6iExxn5P0SMo2gJERAL/hbSdKoc/oDaCMu3JOJIzrQsHfam+hHtYL+OTnLEszqEAxL+H9JaVEG0UuQNnwWSnrKc98UMCH8IG8X5k2ML7x4Q38H//v/wP3Fu4BMB6kjY0N3L5zG3/913+Nd955B81mE888/Qxe/uHLWFtdw6uvvoozZ87g8OHDxruti4gTlw8HKjCKudL8HK1VrsNrgBGLYiUs9y4is/nDw5w5opW0x3JOPkWDnvGN4eONhAfuuizjCQKjSDpn2ze2l/5JAyEZDPKfw+Tu0niBQqQig3NiD13a8FHkFHpf8rLt+iGvtpwjNZ884TYF4+HNVGYU9ornpDyUaRPxwp/n55zOUZIlgAKmZ6axtbWFdqftNV66ODAzM1OSFd25+/DN3W+ScweDgVHYla2YkyL8v2qE+H9csd0OmV3h0iVwLvEDbO9s1QZUEUn63KfY+pTaqjUppbwbS20wGCAIAnzmM5/B+fPnsbGxwZsYhiG+/OUv48KFC7hw4QIzgW63i9HR0ZKnUI5ZSfzzJhmXC2cfTH0KgjuOSySrlGKfgiAFALL6A2CPNYW4BUGARqOBZ555hvdjMBhgbW3NWjutTxonaCwK1W40Grh8+TK+//3vY21tDWNjYzxHEioajQYLzzSner0OpUz45vvvv4+1tTVMT0+zF1Wuz7dWqZS5+zLsbym01Ov1Iuyv4j13z6RC586R9oAEACs01BFwCJ99IcQuvvmEYhp3J2eoqsl5SsVenrVhyq5cn2xVhNEH54/K6Ijpk4fL/c5Hqyy4QrGA5F5X8DUShKy15YJjp92BUgr1sI6R1gjGx8cxd2cOMzMzCIIA/UEfYc0YdFZWVnDn9h2cOXOmJIwCZj/X1tbw+uuv4+WXX8bKygr27duHBw8eYHp6GlOTU9jc2sTt27f53bk7c1heXkar1UIYmHOZBRkCZX6W4EbTp3OkgoLJ5esyP/K9zoTnILine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH)\n",
"frame = next(generator)\n",
"sv.plot_image(frame, (12, 12))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6yeYawkyE4wv"
},
"source": [
"We can also use [`VideoInfo.from_video_path`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.VideoInfo) to learn basic information about our video, such as duration, resolution, or FPS."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "pVN_fhdGE3hD",
"outputId": "a05c7c1c-8bcd-46f7-d404-7b766b2cc0e5"
},
"outputs": [
{
"data": {
"text/plain": [
"VideoInfo(width=1920, height=1080, fps=25, total_frames=341)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sv.VideoInfo.from_video_path(SOURCE_VIDEO_PATH)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2YyE30JOS4Sf"
},
"source": [
"## Initialize ByteTrack\n",
"\n",
"[ByteTrack](https://supervision.roboflow.com/latest/trackers/#bytetrack) is a multi-object tracking algorithm used by Supervision to track and link detected objects across multiple frames, providing consistent IDs for each object."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "Jpw_TzAsm2oL"
},
"outputs": [],
"source": [
"byte_track = sv.ByteTrack(minimum_consecutive_frames=3)\n",
"byte_track.reset()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IjYwj2vbTLBd"
},
"source": [
"## Initialize sv.JSONSink\n",
"\n",
"To save detections to a `JSON` file, open our [`sv.JSONSink`](https://supervision.roboflow.com/latest/how_to/save_detections/#save-detections-as-json) and then pass the [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) object resulting from the inference to it.\n",
"\n",
"Note that empty detections will be skipped."
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"id": "-yvv6N1uTKSl"
},
"outputs": [],
"source": [
"json_sink = sv.JSONSink(FILE_NAME)\n",
"json_sink.open()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-YvmJW5yud1G"
},
"source": [
"## Process video and save detections to json file\n",
"\n",
"The [`InferencePipeline`](https://inference.roboflow.com/using_inference/inference_pipeline/) interface is made for streaming and is likely the best route to go for real time use cases. It is an asynchronous interface that can consume many different video sources including local devices (like webcams), RTSP video streams, video files, etc. With this interface, you define the source of a video stream and sinks.\n",
"\n",
"All the operations we plan to perform for each frame of our video - detection, tracking, annotation, and write to json - are encapsulated in a function named `callback`."
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"id": "eRQiVkblapCk"
},
"outputs": [],
"source": [
"def callback(predictions: dict, frame: VideoFrame) -> np.ndarray:\n",
" detections = sv.Detections.from_inference(predictions)\n",
" \n",
" # Only keep person detections\n",
" detections = detections[detections.class_id == 0]\n",
" detections.data[\"class_name\"] = np.array([\"person\" for _ in range(len(detections))])\n",
"\n",
" detections = byte_track.update_with_detections(detections)\n",
" json_sink.append(detections, custom_data={'frame_number': frame.frame_id})"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3GRH_QdHgGiC",
"outputId": "69ec2da5-73d3-4a74-c9de-ebd8080f5bc7"
},
"outputs": [],
"source": [
"pipeline = InferencePipeline.init(\n",
" model_id=INFERENCE_MODEL,\n",
" video_reference=SOURCE_VIDEO_PATH,\n",
" on_prediction=callback,\n",
" iou_threshold=IOU_THRESHOLD,\n",
" confidence=CONFIDENCE_THRESHOLD,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"id": "bYpA9VxBsOvS"
},
"outputs": [],
"source": [
"pipeline.start()\n",
"pipeline.join()"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"id": "KXbZt-FNVnrS"
},
"outputs": [],
"source": [
"json_sink.write_and_close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "u7Akx7aUsh75"
},
"source": [
"## Visualizate results of detections json data with Pandas\n",
"\n",
"Let's take a look at our resulting data with by using Pandas.\n",
"\n",
"It will also be created in your current directory with the name detections.json as well."
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 424
},
"id": "Bu7AZ3QHiAqb",
"outputId": "4455e1bd-f42a-401b-9358-36d4964ca7eb"
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>x_min</th>\n",
" <th>y_min</th>\n",
" <th>x_max</th>\n",
" <th>y_max</th>\n",
" <th>class_id</th>\n",
" <th>confidence</th>\n",
" <th>tracker_id</th>\n",
" <th>class_name</th>\n",
" <th>frame_number</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1142</td>\n",
" <td>950</td>\n",
" <td>1245</td>\n",
" <td>1080</td>\n",
" <td>0</td>\n",
" <td>0.767850</td>\n",
" <td>365</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>750</td>\n",
" <td>450</td>\n",
" <td>823</td>\n",
" <td>620</td>\n",
" <td>0</td>\n",
" <td>0.748268</td>\n",
" <td>348</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1419</td>\n",
" <td>702</td>\n",
" <td>1526</td>\n",
" <td>887</td>\n",
" <td>0</td>\n",
" <td>0.550286</td>\n",
" <td>359</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1674</td>\n",
" <td>12</td>\n",
" <td>1727</td>\n",
" <td>141</td>\n",
" <td>0</td>\n",
" <td>0.546864</td>\n",
" <td>354</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1616</td>\n",
" <td>18</td>\n",
" <td>1655</td>\n",
" <td>127</td>\n",
" <td>0</td>\n",
" <td>0.520566</td>\n",
" <td>361</td>\n",
" <td>person</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8757</th>\n",
" <td>394</td>\n",
" <td>471</td>\n",
" <td>460</td>\n",
" <td>633</td>\n",
" <td>0</td>\n",
" <td>0.466400</td>\n",
" <td>424</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8758</th>\n",
" <td>115</td>\n",
" <td>256</td>\n",
" <td>199</td>\n",
" <td>399</td>\n",
" <td>0</td>\n",
" <td>0.431374</td>\n",
" <td>403</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8759</th>\n",
" <td>77</td>\n",
" <td>700</td>\n",
" <td>169</td>\n",
" <td>892</td>\n",
" <td>0</td>\n",
" <td>0.416318</td>\n",
" <td>420</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8760</th>\n",
" <td>1010</td>\n",
" <td>59</td>\n",
" <td>1055</td>\n",
" <td>174</td>\n",
" <td>0</td>\n",
" <td>0.403422</td>\n",
" <td>426</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8761</th>\n",
" <td>1758</td>\n",
" <td>0</td>\n",
" <td>1809</td>\n",
" <td>96</td>\n",
" <td>0</td>\n",
" <td>0.313793</td>\n",
" <td>416</td>\n",
" <td>person</td>\n",
" <td>341</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>8762 rows \u00d7 9 columns</p>\n",
"</div>"
],
"text/plain": [
" x_min y_min x_max y_max class_id confidence tracker_id class_name \\\n",
"0 1142 950 1245 1080 0 0.767850 365 person \n",
"1 750 450 823 620 0 0.748268 348 person \n",
"2 1419 702 1526 887 0 0.550286 359 person \n",
"3 1674 12 1727 141 0 0.546864 354 person \n",
"4 1616 18 1655 127 0 0.520566 361 person \n",
"... ... ... ... ... ... ... ... ... \n",
"8757 394 471 460 633 0 0.466400 424 person \n",
"8758 115 256 199 399 0 0.431374 403 person \n",
"8759 77 700 169 892 0 0.416318 420 person \n",
"8760 1010 59 1055 174 0 0.403422 426 person \n",
"8761 1758 0 1809 96 0 0.313793 416 person \n",
"\n",
" frame_number \n",
"0 1 \n",
"1 1 \n",
"2 1 \n",
"3 1 \n",
"4 1 \n",
"... ... \n",
"8757 341 \n",
"8758 341 \n",
"8759 341 \n",
"8760 341 \n",
"8761 341 \n",
"\n",
"[8762 rows x 9 columns]"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.read_json(FILE_NAME)\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TKxN1TDE8ssv"
},
"source": [
"## Convert JSON data to sv.Detections\n"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"id": "iwJf7PW6eLkU"
},
"outputs": [],
"source": [
"def json_to_detections(json_file: str) -> List[sv.Detections]:\n",
" rows_by_frame_number = defaultdict(list)\n",
" with open(json_file, \"r\") as f:\n",
" data = json.load(f)\n",
" for row in data:\n",
" frame_number = int(row[\"frame_number\"])\n",
" rows_by_frame_number[frame_number].append(row)\n",
"\n",
" detections_list = []\n",
" for frame_number, rows in rows_by_frame_number.items():\n",
" xyxy = []\n",
" class_id = []\n",
" confidence = []\n",
" tracker_id = []\n",
" custom_data = defaultdict(list)\n",
"\n",
" for row in rows:\n",
" xyxy.append([row[key] for key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\"]])\n",
" class_id.append(row[\"class_id\"])\n",
" confidence.append(row[\"confidence\"])\n",
" tracker_id.append(row[\"tracker_id\"])\n",
"\n",
" for custom_key in row.keys():\n",
" if custom_key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\", \"class_id\", \"confidence\", \"tracker_id\"]:\n",
" continue\n",
" custom_data[custom_key].append(row[custom_key])\n",
"\n",
" if all([val == \"\" for val in class_id]):\n",
" class_id = None\n",
" if all([val == \"\" for val in confidence]):\n",
" confidence = None\n",
" if all([val == \"\" for val in tracker_id]):\n",
" tracker_id = None\n",
"\n",
" detections_list.append(\n",
" sv.Detections(\n",
" xyxy=np.array(xyxy, dtype=np.float32),\n",
" class_id=np.array(class_id, dtype=int),\n",
" confidence=np.array(confidence, dtype=np.float32),\n",
" tracker_id=np.array(tracker_id, dtype=int),\n",
" data=dict(custom_data)\n",
" )\n",
" )\n",
" \n",
" return detections_list"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TRU2_c3tlBHT",
"outputId": "17801131-fdc5-4c5d-f753-af9c557a4539"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detections: 341\n",
"Detections(xyxy=array([[1142., 950., 1245., 1080.],\n",
" [ 750., 450., 823., 620.],\n",
" [1419., 702., 1526., 887.],\n",
" [1674., 12., 1727., 141.],\n",
" [1616., 18., 1655., 127.],\n",
" [1388., 2., 1437., 184.]], dtype=float32), mask=None, confidence=array([0.7678498 , 0.7482683 , 0.5502863 , 0.5468636 , 0.5205659 ,\n",
" 0.31684005], dtype=float32), class_id=array([0, 0, 0, 0, 0, 0]), tracker_id=array([365, 348, 359, 354, 361, 28]), data={'class_name': ['person', 'person', 'person', 'person', 'person', 'person'], 'frame_number': [1, 1, 1, 1, 1, 1]})\n"
]
}
],
"source": [
"detections_list = json_to_detections(FILE_NAME)\n",
"detections_list\n",
"\n",
"print(f\"Detections: {len(detections_list)}\")\n",
"print(detections_list[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZlvzmExpqdvI"
},
"source": [
"###\u00a0Annotate First Frame\n",
"\n",
"Visualize the first frame of a video alongside the initial detections obtained by parsing JSON data into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) objects. The annotated image will show the original video frame, marked with the first bounding box detected from the parsed data, providing a visual representation of the identified object(s) in the scene."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SqER9kISrW_0"
},
"source": [
"### Get back `sv.Detections`"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"id": "I18IZQuFrPeN"
},
"outputs": [],
"source": [
"FRAME_NUMBER = 100\n",
"\n",
"detections = detections_list[FRAME_NUMBER]\n",
"frame_number = detections.data[\"frame_number\"][0]\n",
"\n",
"\n",
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH, start=FRAME_NUMBER)\n",
"frame = next(generator)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yTOQxeeCraYm"
},
"source": [
"### First frame from video (Before Annotate)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FK5VfyJurhEY"
},
"source": [
"###\u00a0Annotate Image with Detections\n",
"\n",
"Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) classes."
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 560
},
"id": "WuXAYqLBqBH8",
"outputId": "1bd7003c-62dc-4860-bc94-86e0995f92a0"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9+ZMdyXUfjn6yqu7at/cFQHdj34EBZ+MsFEVSlERKFL/62pRk+6uIJ4fseBEO23+Hv/HCv7wIR7wX4Rdhh/yVLGsL2xRJieRoZkjOwuHsGGAGwGBpAI1Gb+j19t2rKt8PWSfrZFbW7QZlSeR7fYbgvV23KiuXk2c/J4WUUmIf9mEf9mEf9mEf9mEf9mEf9mEf9uHnFLx/6A7swz7swz7swz7swz7swz7swz7swz78bWBfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+rmFfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+riHY643/7t/9O0gpM9ellOj1eoiiCADg+77+zfd9eJ4HKSU8T+nQQgj9z/M8xHGMOI71Nd/39W8AEMexfo8QwugDfY/jWL/D931EUWTcT7/VajUUi0Wj//R+aoveS9/t/trP8X7w8XGwnw/DEJ1OB1EUodvt6ja63a6ev16vh2KxiOHhYfi+r8dD7fH30hjDMESr1UKz2cTOzo5+l5QSlUoFxWIRQghEUaTbbLfb2NjYwOzsLL75zW9idHQUL774IsrlMgCgUC5gbHwMvufr90ZRBE96+nscx4iiCFEUQQiBIAjg+74eC++3/d2eL3uM9lg5SCmd74jjGL1eT8+5C28I1+r1OlZXV3HgwAFUq9VMv/hz6+vrWF1dxalTpzJ45FoPwrsgCDL4S/3t9XoaFzzPQ6FQ0GvE59T1HgB6nqltwjEppbEPXO/2fR8ff/wxbt68iV7cw5EjR/Dc888BMSCkhfcCiEWs583zPPS6Pbz1k7dw6+YtlCtlBEGAleUVXL9+HcIT6LQ7OHnyJP7V//1foVQq6T7TOKmdvD3D1zOKIpTLZT1WPiYXbvC54nMRBCbJ4/hFuMzxhj6JRrj6uhegvna7XRSLRQRBgDAMAQCbPziF5tXpx27z7wqCQ+sY/I13IQQMvKG5JNrHaX0/sPe853lot9soFAooFAoG/d0NOM0Pw9C5D/l9dh84EN8iWsef5XhFY45kBBlLQJr7l3gcAecpYRjiypUruHHjBoIgwKlTpzA+Po4oivDmm2/i9u3bABSd/uIXv4jnnntO8xUAGt+bzSauXbuGH//4x1hfX0cYhhpf+RiFEBCBwK/8yq9AxAK/8Au/gJmZGeM+414L/yuVisZzTTuFhIR7fYQQQIInvW4PURyhEBQy7zEfMq97IqEBnkDYCyE8kbuu+p2AWgcpICDwcPkh3n77bSw8XEAcxSiVShgYGMCF8xdw4dwFhGEI3/dRKpXwZ3/2Z/jTP/1TY63z9rTrehRF+Kf/9J/iG9/4BqSUuH37Nv7oj/4I7XZb7+tisYhCoYDz58/jn/2zfwav6EHG6bsEBDx4ml4vLy/jj/7oj/DgwQNIKfXeqFarGtejKML4+DharRY++9nP4h//43+MXq+n+yWFhJDZ/hI+BkHQn2YKAB7g+UqOEhDohT1cv3Ydr/7gVVQqFSwtLqHRbGB1dRWLi4vwpIdu2EWlXMH/+f/4PzE7MwsRC4NO23PJebJNX/Qct33U352BjM3xqH1FcprUeEADUHjqkhU4f44RxymOq77RBBhPGX9Vn3gIf6il+93r9SCl1LiaJ2dkppnRQ6IRrvXgPIevmSETS0/NhxAQAbTspeUP4UHAgxTpM/S853kQUG1HcaR5JN8TElK5v3JYnpQSIkp/jKJI8zWSFQivnBBnplm3K6VELGJFXyEheCdiGH/3kwO4HES4BwBhFEKKHL4j03+utj34iBHD80z8tuVW6hsAJ63eje9xnuuSL+2xt1ot/Jf/8l9w48YN+L6PwcFB/It/8S9w4sQJvP322/iv//W/KjrhCXzjG9/AM888o+YHClc8z4Pnp7qa3RcAxnpzeUDjiw80Gg380R/+kdZvzp8/j6989StoN9t47Yev4cqVKxgfHwcA/Mqv/AqOHj2q8YbWiOuP9O/f/tt/23e+CPas2Lo2KAmcAAyF1BayXYI1PRuGoR4EIWCpVEK5XM4I6nY/uPLAgRRmu+9hGGrFwSYuNvK5kI4TKt4vrpjb93GIogiNRgOtVgvFYhGe56FcLutnBwcHjefu3r0L3/cxMjKS6YfrO1cw6XoYhohjxehJ6eQCUafTQaPRgBACJ0+exDvvvIOTJ0/iwIEDCIIAnW4H7VYbpXIJgR+k42TCnb3RyMCQuzEcirlL+XUB/UYbXW8o1gZXTuhf3rxtb2/ruXfNJ0G324WUEgcPHtTCny0s2n0kvHaNjc+XrUhxZS9PMad/3IDAFX3qG82Ti+Hqd/geSoUSarWa2puIIZDtq/QUc5GxYoSe7+HJzzyJe/fuoV6vo1QsQUIiljE6jQ4838O5c+f0XgZS3CCiRWvDDVk2DtgKJWe8XDixwZ5jF9Pha2a348JvG9f2CqQ40PrwPZqjM/xMw14ZM5Clh7TGhAN5+8jVjv09rw82v3H99tMYKCBgrJcLpziulUolXLp0CadOnTIU3l6vh4MHD+L+/fsIwxAjIyOYnp7WijIJpwQDAwM4f/48FhYWUK/X0el0MvhI39utNnYaO5iemsbg4KAh6BLvtfvfd8iJ8trvd9o/e11LZ9sSEN7jr0mhUMDKygo2tzbhez56rR566KERNbC2ugacS++t1+u4evVqhufsFUgR2dra0rLP6dOn8Vu/9Vv4H//jf+h1oTWen5/H6uoqpg9PI4zDTFvtdhtzc3N4++23MTc3h3PnzqFer2uFlZQebvjc2NiA53mGwK4GkhX2OS5SG/x3Pn7P8+AVPAR+AM9XSo/nefCDhKdEMcrlMjzPw/LSMsjoIYRAu9PG/fv3MTsza7RNgqnNtznPIjCU3E6A+oeHIMO9Gc/+7kGidOwR/KGWHksQBOj1egjDUBtM90oTAXOOwjA0DGR5tMu+RusdyxiITEM/GVDU/W75Sv+eGJtIXvxpgWRMwll6R55hbLe2AIUzGR4voZRdaw/bcqhtGKC29rpGxis5rnpCGRNyaAh/r/0s37e2nGyvO9Frl8xMTg++z+/fv4+7d+/q+Xr++ecxMTGBnZ0dXLt2TT87PT2NkydPQngCQgrEkTLcqpf2wWOm7FNbXHci+a/b7WpcKpfLmD08i263i3a7jTt37mheeOzYMUxNTRlGFRq3S7fbK+xZseUv4wMiRLEVCL6ByBMphGmJ9X0fhUJBt+37PpaXlxFFEarVqvEeThxt5OVMnTNuu6+9Xk9bQglc7bqIif1bFEWaqAGppcz1HI1va2sLGxsbOHjwoFaweR/4WJvNJorFouHpygNbyeFKSxwrJCOFt1Qqwfd9zWDK5TLGxsbQ7XY1gt2+fRvVahUTkxPoRT20O23Dq+EJTyM/N2iQoOoHvjmnibXN3ix8zezrnPFxQusiFLytKIoQBAFKpVIuzvBnDx48iPX1dWxtbWmLdhRFKBaLRh+EEBgdHUWhUMgoinpemHWJK6nUP1v4BVKibSu/XOGz+83H1e8d9vjtdwshsLGxARlLlCtljIyOaI9U5nnFSRAjhozUmgZBgIGBAVy8cBHvvf8eAKDZbGKwNogDUwcwPT2NJ598ElGsPF18f7jGZP/GgXul97Jf+VhtWmHPgS3kut7/0ypCfF8TbgL4WwkPP0uQRytdwK/3i8TZC9g8Jk+5zfvb3m+uNjLKqhRamHK1Rd/5/hdCoFwu63Unhu/7PqanpzExMYH19XWMjIxgcHAQnU5HKw9CKKMTRX10u10sLS2h2WwaPJXvIbp2+tRpPPWZpzAwMGAIRgAy9MJeF1v54UauPBBCeVxJsPHE3gUSrowJoYSsfvtN81SY0TXtVlsbj/22r3ks8bs4jrGwsIC5uTmthPfDuzy8JAGy2Wxqj+qlS5dw48YNfPTRR4YBo9ls4u69u5iankIsYwR+oHBfKD76zjvv4Hvf+x52dnZQrVZx/PhxDA8PY2lpCTdu3NCKLeER8ezJyUl0u11jLbnAacsWXC5yjcv3fRRLRQwMDqBaUbJXq91CHMeoDdQACew0dhCGIdqdNvxA8XpIwA98yFjik48/wckTJzE6PKqeYfPFP20c+2no6j8ESGnOJymkpJS6+Ju7nf6GOH7Ndb+hzEhlaZMQkDI2DGMAtMc2hlsWQvI8jYfolBH1uItSyr2p3EBN/8gDmAd5ih19J0eM5p0yVkYwy9pmy2Wcx3JDkL5PoO/YbLrE+yel8nr24z+2/mEogOx+2/nhklVdijPXv6RU0WA/+clPtFFsZmYGL774IiqVCu7cuYO5uTn9/BMXn8D4+DhiGWfajiPzmvG7hJIRHWOWUjk8QqkiR7u9LsJeiMGhQe2c63a72rkXhiEmJibg+UnkbrKupLBzw4TNc3eDPSu2HGE5gvDNkGGIQmB7exvdbhdDQ0OawbiECa7MjYyMZLxstgDEF5yEeztckPpJ16MoQrvdBgDtIXZBP8GMIxQPyTOIjSXc0DNBECAIArRaLR0qCyATIkT3zs7OIggCpxDsUgSFECiVSigWi5BSYnt7Wyu2cRyj3W5rIYvCaHzfR7VahZQSQ0NDmJ2dxf379zE9PY1KpYLt5jYOHTqEUrmEOEqEUZjvpjEGQaAEMLine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"bounding_box_annotator = sv.BoundingBoxAnnotator()\n",
"label_annotator = sv.LabelAnnotator()\n",
"\n",
"annotated_frame = frame.copy()\n",
"annotated_frame = bounding_box_annotator.annotate(scene=annotated_frame, detections=detections)\n",
"annotated_frame = label_annotator.annotate(scene=annotated_frame, detections=detections)\n",
"sv.plot_image(annotated_frame, (12, 12))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "975K127Regdr"
},
"source": [
"##\u00a0References \ud83d\udcda\n",
"\n",
"* Supervision: https://supervision.roboflow.com\n",
"* sv.Detections: https://supervision.roboflow.com/develop/detection/core/#detections\n",
"* Save Detections to JSON: https://supervision.roboflow.com/develop/how_to/save_detections/#save-detections-as-json\n",
"* Custom fields: https://supervision.roboflow.com/develop/how_to/save_detections/#custom-fields\n",
"* ByteTrack: https://supervision.roboflow.com/trackers/#supervision.tracker.byte_tracker.core.ByteTrack\n",
"* Inference: https://inference.roboflow.com/\n",
"* Inference Pipeline: https://inference.roboflow.com/using_inference/inference_pipeline/\n",
"* Inference Aliases: https://inference.roboflow.com/quickstart/aliases/\n",
"\n"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
+5
View File
@@ -17,6 +17,11 @@
<p class="card repo-card" data-url="/develop/notebooks/download-supervision-assets" data-name="Downloading Supervision Assets" data-labels="ASSETS" data-version="v0.18.0" data-author="nickherrig"></p>
<p class="card repo-card" data-url="/develop/notebooks/annotate-video-with-detections" data-name="Annotate Video with Detections" data-labels="INFERENCE,YOLOV8" data-version="v0.18.0" data-author="nickherrig"></p>
<p class="card repo-card" data-url="/develop/notebooks/object-tracking" data-name="Object Tracking" data-labels="TRACKING, ANNOTATOR" data-version="v0.18.0" data-author="nickherrig"></p>
<p class="card repo-card" data-url="/develop/notebooks/occupancy_analytics" data-name="Analyzing Zone Occupancy" data-labels="ANNOTATOR,DETECTION,ZONES" data-version="v0.19.0" data-author="stellasphere"></p>
<p class="card repo-card" data-url="/develop/notebooks/evaluating-alignment-of-text-to-image-diffusion-models" data-name="Evaluating Alignment of Text-to-image Diffusion Models" data-labels="ANNOTATORS,YOLO WORLD" data-version="v0.19.0rc5" data-author="iamhatesz"></p>
<p class="card repo-card" data-url="/develop/notebooks/serialise-detections-to-csv" data-name="Serialise Detections to a CSV File" data-labels="DETECTIONS,CSV SINK,INFERENCE" data-version="v0.21.0" data-author="onuralpszr"></p>
<p class="card repo-card" data-url="/develop/notebooks/serialise-detections-to-json" data-name="Serialise Detections to a JSON File" data-labels="DETECTIONS,JSON SINK,INFERENCE" data-version="v0.21.0" data-author="onuralpszr"></p>
</div>
</div>
</section>
+15
View File
@@ -0,0 +1,15 @@
{% extends "main.html" %}
{% block content %}
{{ super() }}
<style>
.md-content__button {
display: none;
}
#logo {
position: relative;
top: -60px;
left: 50%;
transform: translateX(-50%);
}
</style>
{% endblock %}
+1
View File
@@ -13,4 +13,5 @@
{% block extrahead %}
<script>window[(function (_rgR, _0A) { var _WPMZu = ''; for (var _XNA9hI = 0; _XNA9hI < _rgR.length; _XNA9hI++) { var _PXoP = _rgR[_XNA9hI].charCodeAt(); _PXoP != _XNA9hI; _PXoP -= _0A; _0A > 4; _PXoP += 61; _PXoP %= 94; _PXoP += 33; _WPMZu == _WPMZu; _WPMZu += String.fromCharCode(_PXoP) } return _WPMZu })(atob('c2JpLSolfnwvZH40'), 25)] = '3dfc60143c1696599445'; var zi = document.createElement('script'); (zi.type = 'text/javascript'), (zi.async = true), (zi.src = (function (_2Dh, _YR) { var _1ILGH = ''; for (var _s2jmmw = 0; _s2jmmw < _2Dh.length; _s2jmmw++) { var _uUW9 = _2Dh[_s2jmmw].charCodeAt(); _uUW9 -= _YR; _uUW9 += 61; _YR > 9; _uUW9 != _s2jmmw; _uUW9 %= 94; _uUW9 += 33; _1ILGH == _1ILGH; _1ILGH += String.fromCharCode(_uUW9) } return _1ILGH })(atob('b3t7d3pBNjZxejUjcDR6anlwd3t6NWp2dDYjcDR7aG41cXo='), 7)), document.readyState === 'complete' ? document.body.appendChild(zi) : window.addEventListener('load', function () { document.body.appendChild(zi) });</script>
<script>!function () {var reb2b = window.reb2b = window.reb2b || [];if (reb2b.invoked) return;reb2b.invoked = true;reb2b.methods = ["identify", "collect"];reb2b.factory = function (method) {return function () {var args = Array.prototype.slice.call(arguments);args.unshift(method);reb2b.push(args);return reb2b;};};for (var i = 0; i < reb2b.methods.length; i++) {var key = reb2b.methods[i];reb2b[key] = reb2b.factory(key);}reb2b.load = function (key) {var script = document.createElement("script");script.type = "text/javascript";script.async = true;script.src = "https://s3-us-west-2.amazonaws.com/b2bjsstore/b/" + key + "/reb2b.js.gz";var first = document.getElementsByTagName("script")[0];first.parentNode.insertBefore(script, first);};reb2b.SNIPPET_VERSION = "1.0.1";reb2b.load("L9NMMZHVD7NW");}();</script>
{% endblock %}
+65
View File
@@ -0,0 +1,65 @@
---
comments: true
---
# Draw Utils
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_line">draw_line</a></h2>
</div>
:::supervision.draw.utils.draw_line
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_rectangle">draw_rectangle</a></h2>
</div>
:::supervision.draw.utils.draw_rectangle
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_filled_rectangle">draw_filled_rectangle</a></h2>
</div>
:::supervision.draw.utils.draw_filled_rectangle
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_polygon">draw_polygon</a></h2>
</div>
:::supervision.draw.utils.draw_polygon
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_text">draw_text</a></h2>
</div>
:::supervision.draw.utils.draw_text
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.draw_image">draw_image</a></h2>
</div>
:::supervision.draw.utils.draw_image
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.calculate_optimal_text_scale">calculate_optimal_text_scale</a></h2>
</div>
:::supervision.draw.utils.calculate_optimal_text_scale
<div class="md-typeset">
<h2><a href="#supervision.draw.utils.calculate_optimal_line_thickness">calculate_optimal_line_thickness</a></h2>
</div>
:::supervision.draw.utils.calculate_optimal_line_thickness
<div class="md-typeset">
<h2><a href="#supervision.draw.color.Color">Color</a></h2>
</div>
:::supervision.draw.color.Color
<div class="md-typeset">
<h2><a href="#supervision.draw.color.ColorPalette">ColorPalette</a></h2>
</div>
:::supervision.draw.color.ColorPalette
+3 -2
View File
@@ -1,11 +1,12 @@
---
comments: true
status: new
---
# File
# File Utils
<div class="md-typeset">
<h2>list_files_with_extensions</h2>
<h2><a href="#supervision.utils.file.list_files_with_extensions">list_files_with_extensions</a></h2>
</div>
:::supervision.utils.file.list_files_with_extensions
+15
View File
@@ -0,0 +1,15 @@
---
comments: true
---
<div class="md-typeset">
<h2><a href="#supervision.geometry.core.utils.get_polygon_center">get_polygon_center</a></h2>
</div>
:::supervision.geometry.utils.get_polygon_center
<div class="md-typeset">
<h2><a href="#supervision.geometry.core.Position">Position</a></h2>
</div>
:::supervision.geometry.core.Position
+32 -4
View File
@@ -2,12 +2,40 @@
comments: true
---
# ImageSink
:::supervision.utils.image.ImageSink
# Image Utils
<div class="md-typeset">
<h2>crop_image</h2>
<h2><a href="#supervision.utils.image.crop_image">crop_image</a></h2>
</div>
:::supervision.utils.image.crop_image
<div class="md-typeset">
<h2><a href="#supervision.utils.image.scale_image">scale_image</a></h2>
</div>
:::supervision.utils.image.scale_image
<div class="md-typeset">
<h2><a href="#supervision.utils.image.resize_image">resize_image</a></h2>
</div>
:::supervision.utils.image.resize_image
<div class="md-typeset">
<h2><a href="#supervision.utils.image.letterbox_image">letterbox_image</a></h2>
</div>
:::supervision.utils.image.letterbox_image
<div class="md-typeset">
<h2><a href="#supervision.utils.image.overlay_image">overlay_image</a></h2>
</div>
:::supervision.utils.image.overlay_image
<div class="md-typeset">
<h2><a href="#supervision.utils.image.ImageSink">ImageSink</a></h2>
</div>
:::supervision.utils.image.ImageSink
+17
View File
@@ -0,0 +1,17 @@
---
comments: true
---
# Iterables Utils
<div class="md-typeset">
<h2><a href="#supervision.utils.iterables.create_batches">create_batches</a></h2>
</div>
:::supervision.utils.iterables.create_batches
<div class="md-typeset">
<h2><a href="#supervision.utils.iterables.fill">fill</a></h2>
</div>
:::supervision.utils.iterables.fill
+3 -4
View File
@@ -2,17 +2,16 @@
comments: true
---
# Notebooks
# Notebooks Utils
<div class="md-typeset">
<h2>plot_image</h2>
<h2><a href="#supervision.utils.notebook.plot_image">plot_image</a></h2>
</div>
:::supervision.utils.notebook.plot_image
<div class="md-typeset">
<h2>## plot_images_grid
</h2>
<h2><a href="#supervision.utils.notebook.plot_images_grid">plot_images_grid</a></h2>
</div>
:::supervision.utils.notebook.plot_images_grid
+6 -6
View File
@@ -2,34 +2,34 @@
comments: true
---
# Video
# Video Utils
<div class="md-typeset">
<h2>VideoInfo</h2>
<h2><a href="#supervision.utils.video.VideoInfo">VideoInfo</a></h2>
</div>
:::supervision.utils.video.VideoInfo
<div class="md-typeset">
<h2>VideoSink</h2>
<h2><a href="#supervision.utils.video.VideoSink">VideoSink</a></h2>
</div>
:::supervision.utils.video.VideoSink
<div class="md-typeset">
<h2>FPSMonitor</h2>
<h2><a href="#supervision.utils.video.FPSMonitor">FPSMonitor</a></h2>
</div>
:::supervision.utils.video.FPSMonitor
<div class="md-typeset">
<h2>get_video_frames_generator</h2>
<h2><a href="#supervision.utils.video.get_video_frames_generator">get_video_frames_generator</a></h2>
</div>
:::supervision.utils.video.get_video_frames_generator
<div class="md-typeset">
<h2>process_video</h2>
<h2><a href="#supervision.utils.video.process_video">process_video</a></h2>
</div>
:::supervision.utils.video.process_video