Merge branch 'develop' into dfreilich/add-icon-annotator
This commit is contained in:
commit
cfd079bbcb
158 files changed
+23506
-4763
No files matched your search
+2
-2
@@ -19,13 +19,13 @@ as an extra within the Supervision package.
|
||||
```
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>download_assets</h2>
|
||||
<h2><a href="#supervision.assets.downloader.download_assets.download_assets">download_assets</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.assets.downloader.download_assets
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>VideoAssets</h2>
|
||||
<h2><a href="#supervision.assets.downloader.download_assets.VideoAssets">VideoAssets</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.assets.list.VideoAssets
|
||||
+376
-5
@@ -1,6 +1,380 @@
|
||||
### 0.22.0 <small>Jul 12, 2024</small>
|
||||
|
||||
- Added [#1326](https://github.com/roboflow/supervision/pull/1326): [`sv.DetectionsDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset) and [`sv.ClassificationDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.ClassificationDataset) allowing to load the images into memory only when necessary (lazy loading).
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
Constructing `DetectionDataset` with parameter `images` as `Dict[str, np.ndarray]` is deprecated and will be removed in `supervision-0.26.0`. Please pass a list of paths `List[str]` instead.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
The `DetectionDataset.images` property is deprecated and will be removed in `supervision-0.26.0`. Please loop over images with `for path, image, annotation in dataset:`, as that does not require loading all images into memory.
|
||||
|
||||
```python
|
||||
import roboflow
|
||||
from roboflow import Roboflow
|
||||
import supervision as sv
|
||||
|
||||
roboflow.login()
|
||||
rf = Roboflow()
|
||||
|
||||
project = rf.workspace(<WORKSPACE_ID>).project(<PROJECT_ID>)
|
||||
dataset = project.version(<PROJECT_VERSION>).download("coco")
|
||||
|
||||
ds_train = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f"{dataset.location}/train",
|
||||
annotations_path=f"{dataset.location}/train/_annotations.coco.json",
|
||||
)
|
||||
|
||||
path, image, annotation = ds_train[0]
|
||||
# loads image on demand
|
||||
|
||||
for path, image, annotation in ds_train:
|
||||
# loads image on demand
|
||||
```
|
||||
|
||||
- Added [#1296](https://github.com/roboflow/supervision/pull/1296): [`sv.Detections.from_lmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_lmm) now supports parsing results from the [Florence 2](https://huggingface.co/microsoft/Florence-2-large) model, extending the capability to handle outputs from this Large Multimodal Model (LMM). This includes detailed object detection, OCR with region proposals, segmentation, and more. Find out more in our [Colab notebook](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-finetune-florence-2-on-detection-dataset.ipynb).
|
||||
|
||||
- Added [#1232](https://github.com/roboflow/supervision/pull/1232) to support keypoint detection with Mediapipe. Both [legacy](https://colab.research.google.com/github/googlesamples/mediapipe/blob/main/examples/pose_landmarker/python/%5BMediaPipe_Python_Tasks%5D_Pose_Landmarker.ipynb) and [modern](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker/python) pipelines are supported. See [`sv.KeyPoints.from_mediapipe`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_mediapipe) for more.
|
||||
|
||||
- Added [#1316](https://github.com/roboflow/supervision/pull/1316): [`sv.KeyPoints.from_mediapipe`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_mediapipe) extended to support FaceMesh from Mediapipe. This enhancement allows for processing both face landmarks from `FaceLandmarker`, and legacy results from `FaceMesh`.
|
||||
|
||||
- Added [#1310](https://github.com/roboflow/supervision/pull/1310): [`sv.KeyPoints.from_detectron2`](https://supervision.roboflow.com/latest/keypoint/core/#supervision.keypoint.core.KeyPoints.from_detectron2) is a new `KeyPoints` method, adding support for extracting keypoints from the popular [Detectron 2](https://github.com/facebookresearch/detectron2) platform.
|
||||
|
||||
- Added [#1300](https://github.com/roboflow/supervision/pull/1300): [`sv.Detections.from_detectron2`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_detectron2) now supports segmentation models detectron2. The resulting masks can be used with [`sv.MaskAnnotator`](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.MaskAnnotator) for displaying annotations.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
from detectron2 import model_zoo
|
||||
from detectron2.engine import DefaultPredictor
|
||||
from detectron2.config import get_cfg
|
||||
import cv2
|
||||
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
cfg = get_cfg()
|
||||
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
|
||||
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
|
||||
predictor = DefaultPredictor(cfg)
|
||||
|
||||
result = predictor(image)
|
||||
detections = sv.Detections.from_detectron2(result)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
annotated_frame = mask_annotator.annotate(scene=image.copy(), detections=detections)
|
||||
```
|
||||
|
||||
- Added [#1277](https://github.com/roboflow/supervision/pull/1277): if you provide a font that supports symbols of a language, [`sv.RichLabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator.annotate) will draw them on your images.
|
||||
- Various other annotators have been revised to ensure proper in-place functionality when used with `numpy` arrays. Additionally, we fixed a bug where `sv.ColorAnnotator` was filling boxes with solid color when used in-place.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
import
|
||||
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
rich_label_annotator = sv.RichLabelAnnotator(font_path=<TTF_FONT_PATH>)
|
||||
annotated_image = rich_label_annotator.annotate(scene=image.copy(), detections=detections)
|
||||
```
|
||||
|
||||
- Added [#1227](https://github.com/roboflow/supervision/pull/1227): Added support for loading Oriented Bounding Boxes dataset in YOLO format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
train_ds = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path="/content/dataset/train/images",
|
||||
annotations_directory_path="/content/dataset/train/labels",
|
||||
data_yaml_path="/content/dataset/data.yaml",
|
||||
is_obb=True
|
||||
)
|
||||
|
||||
_, image, detections in train_ds[0]
|
||||
|
||||
obb_annotator = OrientedBoxAnnotator()
|
||||
annotated_image = obb_annotator.annotate(scene=image.copy(), detections=detections)
|
||||
```
|
||||
|
||||
- Fixed [#1312](https://github.com/roboflow/supervision/pull/1312): Fixed [`CropAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.TraceAnnotator.annotate).
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`BoxAnnotator` was removed, however `BoundingBoxAnnotator` has been renamed to `BoxAnnotator`. Use a combination of [`BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator) and [`LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) to simulate old `BoundingBox` behavior.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
The name `BoundingBoxAnnotator` has been deprecated and will be removed in `supervision-0.26.0`. It has been renamed to [`BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator).
|
||||
|
||||
- Added [#975](https://github.com/roboflow/supervision/pull/975) 📝 New Cookbooks: serialize detections into [json](https://github.com/roboflow/supervision/blob/de896189b83a1f9434c0a37dd9192ee00d2a1283/docs/notebooks/serialise-detections-to-json.ipynb) and [csv](https://github.com/roboflow/supervision/blob/de896189b83a1f9434c0a37dd9192ee00d2a1283/docs/notebooks/serialise-detections-to-csv.ipynb).
|
||||
|
||||
- Added [#1290](https://github.com/roboflow/supervision/pull/1290): Mostly an internal change, our file utility function now support both `str` and `pathlib` paths.
|
||||
|
||||
- Added [#1340](https://github.com/roboflow/supervision/pull/1340): Two new methods for converting between bounding box formats - [`xywh_to_xyxy`](https://supervision.roboflow.com/latest/detection/utils/#supervision.detection.utils.xywh_to_xyxy) and [`xcycwh_to_xyxy`](https://supervision.roboflow.com/latest/detection/utils/#supervision.detection.utils.xcycwh_to_xyxy)
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`from_roboflow` method has been removed due to deprecation. Use [from_inference](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_inference) instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`Color.white()` has been removed due to deprecation. Use `color.WHITE` instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`Color.black()` has been removed due to deprecation. Use `color.BLACK` instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`Color.red()` has been removed due to deprecation. Use `color.RED` instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`Color.green()` has been removed due to deprecation. Use `color.GREEN` instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`Color.blue()` has been removed due to deprecation. Use `color.BLUE` instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`ColorPalette.default()` has been removed due to deprecation. Use [ColorPalette.DEFAULT](https://supervision.roboflow.com/latest/utils/draw/#supervision.draw.color.ColorPalette.DEFAULT) instead.
|
||||
|
||||
!!! failure "Removed"
|
||||
|
||||
`FPSMonitor.__call__` has been removed due to deprecation. Use the attribute [FPSMonitor.fps](https://supervision.roboflow.com/latest/utils/video/#supervision.utils.video.FPSMonitor.fps) instead.
|
||||
|
||||
### 0.21.0 <small>Jun 5, 2024</small>
|
||||
|
||||
- Added [#500](https://github.com/roboflow/supervision/pull/500): [`sv.Detections.with_nmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.with_nmm) to perform non-maximum merging on the current set of object detections.
|
||||
|
||||
- Added [#1221](https://github.com/roboflow/supervision/pull/1221): [`sv.Detections.from_lmm`](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections.from_lmm) allowing to parse Large Multimodal Model (LMM) text result into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/) object. For now `from_lmm` supports only [PaliGemma](https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/how-to-finetune-paligemma-on-detection-dataset.ipynb) result parsing.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
paligemma_result = "<loc0256><loc0256><loc0768><loc0768> cat"
|
||||
detections = sv.Detections.from_lmm(
|
||||
sv.LMM.PALIGEMMA,
|
||||
paligemma_result,
|
||||
resolution_wh=(1000, 1000),
|
||||
classes=['cat', 'dog']
|
||||
)
|
||||
detections.xyxy
|
||||
# array([[250., 250., 750., 750.]])
|
||||
|
||||
detections.class_id
|
||||
# array([0])
|
||||
```
|
||||
|
||||
- Added [#1236](https://github.com/roboflow/supervision/pull/1236): [`sv.VertexLabelAnnotator`](https://supervision.roboflow.com/latest/keypoint/annotators/#supervision.keypoint.annotators.EdgeAnnotator.annotate) allowing to annotate every vertex of a keypoint skeleton with custom text and color.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
key_points = sv.KeyPoints(...)
|
||||
|
||||
edge_annotator = sv.EdgeAnnotator(
|
||||
color=sv.Color.GREEN,
|
||||
thickness=5
|
||||
)
|
||||
annotated_frame = edge_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
key_points=key_points
|
||||
)
|
||||
```
|
||||
|
||||
- Added [#1147](https://github.com/roboflow/supervision/pull/1147): [`sv.KeyPoints.from_inference`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints.from_inference) allowing to create [`sv.KeyPoints`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints) from [Inference](https://github.com/roboflow/inference) result.
|
||||
|
||||
- Added [#1138](https://github.com/roboflow/supervision/pull/1138): [`sv.KeyPoints.from_yolo_nas`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints.from_yolo_nas) allowing to create [`sv.KeyPoints`](https://supervision.roboflow.com/develop/keypoint/core/#supervision.keypoint.core.KeyPoints) from [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) result.
|
||||
|
||||
- Added [#1163](https://github.com/roboflow/supervision/pull/1163): [`sv.mask_to_rle`](https://supervision.roboflow.com/develop/datasets/utils/#supervision.dataset.utils.rle_to_mask) and [`sv.rle_to_mask`](https://supervision.roboflow.com/develop/datasets/utils/#supervision.dataset.utils.rle_to_mask) allowing for easy conversion between mask and rle formats.
|
||||
|
||||
- Changed [#1236](https://github.com/roboflow/supervision/pull/1236): [`sv.InferenceSlicer`](https://supervision.roboflow.com/develop/detection/tools/inference_slicer/) allowing to select overlap filtering strategy (`NONE`, `NON_MAX_SUPPRESSION` and `NON_MAX_MERGE`).
|
||||
|
||||
- Changed [#1178](https://github.com/roboflow/supervision/pull/1178): [`sv.InferenceSlicer`](https://supervision.roboflow.com/develop/detection/tools/inference_slicer/) adding instance segmentation model support.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8x-seg-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
results = model.infer(image_slice)[0]
|
||||
return sv.Detections.from_inference(results)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
- Changed [#1228](https://github.com/roboflow/supervision/pull/1228): [`sv.LineZone`](https://supervision.roboflow.com/develop/detection/tools/line_zone/) making it 10-20 times faster, depending on the use case.
|
||||
|
||||
- Changed [#1163](https://github.com/roboflow/supervision/pull/1163): [`sv.DetectionDataset.from_coco`](https://supervision.roboflow.com/develop/datasets/core/#supervision.dataset.core.DetectionDataset.from_coco) and [`sv.DetectionDataset.as_coco`](https://supervision.roboflow.com/develop/datasets/core/#supervision.dataset.core.DetectionDataset.as_coco) adding support for run-length encoding (RLE) mask format.
|
||||
|
||||
### 0.20.0 <small>April 24, 2024</small>
|
||||
|
||||
- Added [#1128](https://github.com/roboflow/supervision/pull/1128): [`sv.KeyPoints`](/0.20.0/keypoint/core/#supervision.keypoint.core.KeyPoints) to provide initial support for pose estimation and broader keypoint detection models.
|
||||
|
||||
- Added [#1128](https://github.com/roboflow/supervision/pull/1128): [`sv.EdgeAnnotator`](/0.20.0/keypoint/annotators/#supervision.keypoint.annotators.EdgeAnnotator) and [`sv.VertexAnnotator`](/0.20.0/keypoint/annotators/#supervision.keypoint.annotators.VertexAnnotator) to enable rendering of results from keypoint detection models.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
model = YOLO('yolov8l-pose')
|
||||
|
||||
result = model(image, verbose=False)[0]
|
||||
keypoints = sv.KeyPoints.from_ultralytics(result)
|
||||
|
||||
edge_annotators = sv.EdgeAnnotator(color=sv.Color.GREEN, thickness=5)
|
||||
annotated_image = edge_annotators.annotate(image.copy(), keypoints)
|
||||
```
|
||||
|
||||
- Changed [#1037](https://github.com/roboflow/supervision/pull/1037): [`sv.LabelAnnotator`](/0.20.0/annotators/#supervision.annotators.core.LabelAnnotator) by adding an additional `corner_radius` argument that allows for rounding the corners of the bounding box.
|
||||
|
||||
- Changed [#1109](https://github.com/roboflow/supervision/pull/1109): [`sv.PolygonZone`](/0.20.0/detection/tools/polygon_zone/#supervision.detection.tools.polygon_zone.PolygonZone) such that the `frame_resolution_wh` argument is no longer required to initialize `sv.PolygonZone`.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
The `frame_resolution_wh` parameter in `sv.PolygonZone` is deprecated and will be removed in `supervision-0.24.0`.
|
||||
|
||||
- Changed [#1084](https://github.com/roboflow/supervision/pull/1084): [`sv.get_polygon_center`](/0.20.0/utils/geometry/#supervision.geometry.core.utils.get_polygon_center) to calculate a more accurate polygon centroid.
|
||||
|
||||
- Changed [#1069](https://github.com/roboflow/supervision/pull/1069): [`sv.Detections.from_transformers`](/0.20.0/detection/core/#supervision.detection.core.Detections.from_transformers) by adding support for Transformers segmentation models and extract class names values.
|
||||
|
||||
```python
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForSegmentation
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50-panoptic")
|
||||
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_segmentation(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results, id2label=model.config.id2label)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER)
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
- Fixed [#787](https://github.com/roboflow/supervision/pull/787): [`sv.ByteTrack.update_with_detections`](/0.20.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack.update_with_detections) which was removing segmentation masks while tracking. Now, `ByteTrack` can be used alongside segmentation models.
|
||||
|
||||
### 0.19.0 <small>March 15, 2024</small>
|
||||
|
||||
- Added [#818](https://github.com/roboflow/supervision/pull/818): [`sv.CSVSink`](/0.19.0/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) allowing for the straightforward saving of image, video, or stream inference results in a `.csv` file.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO(<SOURCE_MODEL_PATH>)
|
||||
csv_sink = sv.CSVSink(<RESULT_CSV_FILE_PATH>)
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with csv_sink:
|
||||
for frame in frames_generator:
|
||||
result = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(result)
|
||||
csv_sink.append(detections, custom_data={<CUSTOM_LABEL>:<CUSTOM_DATA>})
|
||||
```
|
||||
|
||||
- Added [#819](https://github.com/roboflow/supervision/pull/819): [`sv.JSONSink`](/0.19.0/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink) allowing for the straightforward saving of image, video, or stream inference results in a `.json` file.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO(<SOURCE_MODEL_PATH>)
|
||||
json_sink = sv.JSONSink(<RESULT_JSON_FILE_PATH>)
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with json_sink:
|
||||
for frame in frames_generator:
|
||||
result = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(result)
|
||||
json_sink.append(detections, custom_data={<CUSTOM_LABEL>:<CUSTOM_DATA>})
|
||||
```
|
||||
|
||||
- Added [#847](https://github.com/roboflow/supervision/pull/847): [`sv.mask_iou_batch`](/0.19.0/detection/utils/#supervision.detection.utils.mask_iou_batch) allowing to compute Intersection over Union (IoU) of two sets of masks.
|
||||
|
||||
- Added [#847](https://github.com/roboflow/supervision/pull/847): [`sv.mask_non_max_suppression`](/0.19.0/detection/utils/#supervision.detection.utils.mask_non_max_suppression) allowing to perform Non-Maximum Suppression (NMS) on segmentation predictions.
|
||||
|
||||
- Added [#888](https://github.com/roboflow/supervision/pull/888): [`sv.CropAnnotator`](/0.19.0/annotators/#supervision.annotators.core.CropAnnotator) allowing users to annotate the scene with scaled-up crops of detections.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
|
||||
result = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(result)
|
||||
|
||||
crop_annotator = sv.CropAnnotator()
|
||||
annotated_frame = crop_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections
|
||||
)
|
||||
```
|
||||
|
||||
- Changed [#827](https://github.com/roboflow/supervision/pull/827): [`sv.ByteTrack.reset`](/0.19.0/tracking/#supervision.tracking.ByteTrack.reset) allowing users to clear trackers state, enabling the processing of multiple video files in sequence.
|
||||
|
||||
- Changed [#802](https://github.com/roboflow/supervision/pull/802): [`sv.LineZoneAnnotator`](/0.19.0/detection/tools/line_zone/#supervision.detection.line_zone.LineZone) allowing to hide in/out count using `display_in_count` and `display_out_count` properties.
|
||||
|
||||
- Changed [#787](https://github.com/roboflow/supervision/pull/787): [`sv.ByteTrack`](/0.19.0/tracking/#supervision.tracking.ByteTrack) input arguments and docstrings updated to improve readability and ease of use.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
The `track_buffer`, `track_thresh`, and `match_thresh` parameters in `sv.ByterTrack` are deprecated and will be removed in `supervision-0.23.0`. Use `lost_track_buffer,` `track_activation_threshold`, and `minimum_matching_threshold` instead.
|
||||
|
||||
- Changed [#910](https://github.com/roboflow/supervision/pull/910): [`sv.PolygonZone`](/0.19.0/detection/tools/polygon_zone/#supervision.detection.tools.polygon_zone.PolygonZone) to now accept a list of specific box anchors that must be in zone for a detection to be counted.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
The `triggering_position ` parameter in `sv.PolygonZone` is deprecated and will be removed in `supervision-0.23.0`. Use `triggering_anchors` instead.
|
||||
|
||||
- Changed [#875](https://github.com/roboflow/supervision/pull/875): annotators adding support for Pillow images. All supervision Annotators can now accept an image as either a numpy array or a Pillow Image. They automatically detect its type, draw annotations, and return the output in the same format as the input.
|
||||
|
||||
- Fixed [#944](https://github.com/roboflow/supervision/pull/944): [`sv.DetectionsSmoother`](/0.19.0/detection/tools/smoother/#supervision.detection.tools.smoother.DetectionsSmoother) removing `tracking_id` from `sv.Detections`.
|
||||
|
||||
### 0.18.0 <small>January 25, 2024</small>
|
||||
|
||||
- Added [#633](https://github.com/roboflow/supervision/pull/720): [`sv.PercentageBarAnnotator`](/0.18.0/annotators/#percentagebarannotator) allowing to annotate images and videos with percentage values representing confidence or other custom property.
|
||||
- Added [#720](https://github.com/roboflow/supervision/pull/720): [`sv.PercentageBarAnnotator`](/0.18.0/annotators/#percentagebarannotator) allowing to annotate images and videos with percentage values representing confidence or other custom property.
|
||||
|
||||
```python
|
||||
>>> import supervision as sv
|
||||
@@ -58,7 +432,6 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
|
||||
|
||||
`sv.ColorPalette.default()` is deprecated and will be removed in `supervision-0.22.0`. Use `sv.ColorPalette.DEFAULT` instead.
|
||||
|
||||
|
||||
- Changed [#769](https://github.com/roboflow/supervision/pull/769): [`sv.ColorPalette.DEFAULT`](/0.18.0/draw/color/#colorpalette) value, giving users a more extensive set of annotation colors.
|
||||
|
||||
- Changed [#677](https://github.com/roboflow/supervision/pull/677): `sv.Detections.from_roboflow` to [`sv.Detections.from_inference`](/0.18.0/detection/core/#supervision.detection.core.Detections.from_inference) streamlining its functionality to be compatible with both the both [inference](https://github.com/roboflow/inference) pip package and the Robloflow [hosted API](https://docs.roboflow.com/deploy/hosted-api).
|
||||
@@ -67,7 +440,6 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
|
||||
|
||||
`Detections.from_roboflow()` is deprecated and will be removed in `supervision-0.22.0`. Use `Detections.from_inference` instead.
|
||||
|
||||
|
||||
- Fixed [#735](https://github.com/roboflow/supervision/pull/735): [`sv.LineZone`](/0.18.0/detection/tools/line_zone/#linezone) functionality to accurately update the counter when an object crosses a line from any direction, including from the side. This enhancement enables more precise tracking and analytics, such as calculating individual in/out counts for each lane on the road.
|
||||
|
||||
### 0.17.0 <small>December 06, 2023</small>
|
||||
@@ -160,13 +532,12 @@ ColorPalette(colors=[Color(r=68, g=1, b=84), Color(r=59, g=82, b=139), ...])
|
||||
|
||||
- Fixed [#477](https://github.com/roboflow/supervision/pull/477): Poetry env definition allowing proper local installation.
|
||||
|
||||
- Fixed [#430](https://github.com/roboflow/supervision/pull/430): [`sv.ByteTrack`](/0.16.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack) to return `np.array([], dtype=int)` when `svDetections` is empty.
|
||||
- Fixed [#430](https://github.com/roboflow/supervision/pull/430): [`sv.ByteTrack`](/0.16.0/trackers/#supervision.tracker.byte_tracker.core.ByteTrack) to return `np.array([], dtype=int)` when `svDetections` is empty.
|
||||
|
||||
!!! failure "Deprecated"
|
||||
|
||||
`sv.Detections.from_yolov8` and `sv.Classifications.from_yolov8` as those are now replaced by [`sv.Detections.from_ultralytics`](/0.16.0/detection/core/#supervision.detection.core.Detections.from_ultralytics) and [`sv.Classifications.from_ultralytics`](/0.16.0/classification/core/#supervision.classification.core.Classifications.from_ultralytics).
|
||||
|
||||
|
||||
### 0.15.0 <small>October 5, 2023</small>
|
||||
|
||||
- Added [#170](https://github.com/roboflow/supervision/pull/170): [`sv.BoundingBoxAnnotator`](/0.15.0/annotators/#supervision.annotators.core.BoundingBoxAnnotator) allowing to annotate images and videos with bounding boxes.
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
template: cookbooks.html
|
||||
comments: true
|
||||
status: new
|
||||
hide:
|
||||
- navigation
|
||||
- toc
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
# Datasets
|
||||
@@ -0,0 +1,17 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Datasets Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.dataset.utils.rle_to_mask">rle_to_mask</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.dataset.utils.rle_to_mask
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.dataset.utils.mask_to_rle">mask_to_rle</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.dataset.utils.mask_to_rle
|
||||
+21
-10
@@ -3,14 +3,25 @@ comments: true
|
||||
status: deprecated
|
||||
---
|
||||
|
||||
These features are phased out due to better alternatives or potential issues in future versions. Deprecated functionalities are supported for **three subsequent releases**, providing time for users to transition to updated methods.
|
||||
# Deprecated
|
||||
|
||||
- [`Detections.from_froboflow`](detection/core.md/#supervision.detection.core.Detections.from_roboflow) is deprecated and will be removed in `supervision-0.22.0`. Use [`Detections.from_inference`](detection/core.md/#supervision.detection.core.Detections.from_inference) instead.
|
||||
- `Color.white()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.WHITE` instead.
|
||||
- `Color.black()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.BLACK` instead.
|
||||
- `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
|
||||
- `Color.green()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.GREEN` instead.
|
||||
- `Color.blue()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.BLUE` instead.
|
||||
- [`ColorPalette.default()`](draw/color.md/#supervision.draw.color.ColorPalette.default) is deprecated and will be removed in `supervision-0.22.0`. Use [`ColorPalette.DEFAULT`](draw/color.md/#supervision.draw.color.ColorPalette.DEFAULT) instead.
|
||||
- `BoxAnnotator` is deprecated and will be removed in `supervision-0.22.0`. Use [`BoundingBoxAnnotator`](annotators.md/#supervision.annotators.core.BoundingBoxAnnotator) and [`LabelAnnotator`](annotators.md/#supervision.annotators.core.LabelAnnotator) instead.
|
||||
- [`FPSMonitor.__call__`](utils/video.md/#supervision.utils.video.FPSMonitor.__call__) is deprecated and will be removed in `supervision-0.22.0`. Use [`FPSMonitor.fps`](utils/video.md/#supervision.utils.video.FPSMonitor.fps) instead.
|
||||
These features are phased out due to better alternatives or potential issues in future versions. Deprecated functionalities are supported for **five subsequent releases**, providing time for users to transition to updated methods.
|
||||
|
||||
- The `track_buffer`, `track_thresh`, and `match_thresh` parameters in [`ByterTrack`](trackers.md/#supervision.tracker.byte_tracker.core.ByteTrack) are deprecated and will be removed in `supervision-0.23.0`. Use `lost_track_buffer,` `track_activation_threshold`, and `minimum_matching_threshold` instead.
|
||||
- The `triggering_position ` parameter in [`sv.PolygonZone`](detection/tools/polygon_zone.md/#supervision.detection.tools.polygon_zone.PolygonZone) will be removed in `supervision-0.23.0`. Use `triggering_anchors ` instead.
|
||||
- The `frame_resolution_wh ` parameter in [`sv.PolygonZone`](detection/tools/polygon_zone.md/#supervision.detection.tools.polygon_zone.PolygonZone) will be removed in `supervision-0.24.0`.
|
||||
- Constructing `DetectionDataset` and `ClassificationDataset` with parameter `images` as `Dict[str, np.ndarray]` will be removed in `supervision-0.26.0`. Please pass a list of paths `List[str]` instead.
|
||||
- The `DetectionDataset.images` property will be removed in `supervision-0.26.0`. Please loop over images with `for path, image, annotation in dataset:`, as that does not require loading all images into memory.
|
||||
- `BoundingBoxAnnotator` has been renamed to `BoxAnnotator` after the old implementation of `BoxAnnotator` has been removed. `BoundingBoxAnnotator` will be removed in `supervision-0.26.0`.
|
||||
|
||||
# Removed
|
||||
|
||||
- [`Detections.from_froboflow`](detection/core.md/#supervision.detection.core.Detections.from_roboflow) is removed as of `supervision-0.22.0`. Use [`Detections.from_inference`](detection/core.md/#supervision.detection.core.Detections.from_inference) instead.
|
||||
- The method `Color.white()` was removed as of `supervision-0.22.0`. Use the constant `Color.WHITE` instead.
|
||||
- The method `Color.black()` was removed as of `supervision-0.22.0`. Use the constant `Color.BLACK` instead.
|
||||
- The method `Color.red()` was removed as of `supervision-0.22.0`. Use the constant `Color.RED` instead.
|
||||
- The method `Color.green()` was removed as of `supervision-0.22.0`. Use the constant `Color.GREEN` instead.
|
||||
- The method `Color.blue()` was removed as of `supervision-0.22.0`. Use the constant `Color.BLUE` instead.
|
||||
- The method [`ColorPalette.default()`](draw/color.md/#supervision.draw.color.ColorPalette.default) was removed as of `supervision-0.22.0`. Use the constant [`ColorPalette.DEFAULT`](draw/color.md/#supervision.draw.color.ColorPalette.DEFAULT) instead.
|
||||
- `BoxAnnotator` was removed as of `supervision-0.22.0`, however `BoundingBoxAnnotator` was immediately renamed to `BoxAnnotator`. Use [`BoxAnnotator`](detection/annotators.md/#supervision.annotators.core.BoxAnnotator) and [`LabelAnnotator`](detection/annotators.md/#supervision.annotators.core.LabelAnnotator) instead of the old `BoxAnnotator`.
|
||||
- The method [`FPSMonitor.__call__`](utils/video.md/#supervision.utils.video.FPSMonitor.__call__) was removed as of `supervision-0.22.0`. Use the attribute [`FPSMonitor.fps`](utils/video.md/#supervision.utils.video.FPSMonitor.fps) instead.
|
||||
@@ -5,7 +5,7 @@ status: new
|
||||
|
||||
# Annotators
|
||||
|
||||
=== "BoundingBox"
|
||||
=== "Box"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
@@ -13,8 +13,8 @@ status: new
|
||||
image = ...
|
||||
detections = sv.Detections(...)
|
||||
|
||||
bounding_box_annotator = sv.BoundingBoxAnnotator()
|
||||
annotated_frame = bounding_box_annotator.annotate(
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
annotated_frame = box_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections
|
||||
)
|
||||
@@ -260,15 +260,22 @@ status: new
|
||||
=== "Label"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
detections = sv.Detections(...)
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER)
|
||||
annotated_frame = label_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections
|
||||
detections=detections,
|
||||
labels=labels
|
||||
)
|
||||
```
|
||||
|
||||
@@ -278,6 +285,52 @@ status: new
|
||||
|
||||
</div>
|
||||
|
||||
=== "RichLabel"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
detections = sv.Detections(...)
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
rich_label_annotator = sv.RichLabelAnnotator(
|
||||
font_path="<TTF_FONT_PATH>",
|
||||
text_position=sv.Position.CENTER
|
||||
)
|
||||
annotated_frame = rich_label_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections,
|
||||
labels=labels
|
||||
)
|
||||
```
|
||||
|
||||
<div class="result" markdown>
|
||||
|
||||
{ align=center width="800" }
|
||||
|
||||
</div>
|
||||
|
||||
=== "Crop"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
detections = sv.Detections(...)
|
||||
|
||||
crop_annotator = sv.CropAnnotator()
|
||||
annotated_frame = crop_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections
|
||||
)
|
||||
```
|
||||
|
||||
=== "Blur"
|
||||
|
||||
```python
|
||||
@@ -380,11 +433,32 @@ status: new
|
||||
|
||||
</div>
|
||||
|
||||
=== "Background Color"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
detections = sv.Detections(...)
|
||||
|
||||
background_overlay_annotator = sv.BackgroundOverlayAnnotator()
|
||||
annotated_frame = background_overlay_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
detections=detections
|
||||
)
|
||||
```
|
||||
|
||||
<div class="result" markdown>
|
||||
|
||||

|
||||
|
||||
</div>
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.BoundingBoxAnnotator">BoundingBoxAnnotator</a></h2>
|
||||
<h2><a href="#supervision.annotators.core.BoxAnnotator">BoxAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.annotators.core.BoundingBoxAnnotator
|
||||
:::supervision.annotators.core.BoxAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.RoundBoxAnnotator">RoundBoxAnnotator</a></h2>
|
||||
@@ -470,6 +544,18 @@ status: new
|
||||
|
||||
:::supervision.annotators.core.LabelAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.RichLabelAnnotator">RichLabelAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.annotators.core.RichLabelAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.IconAnnotator">IconAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.annotators.core.IconAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.BlurAnnotator">BlurAnnotator</a></h2>
|
||||
</div>
|
||||
@@ -489,10 +575,16 @@ status: new
|
||||
:::supervision.annotators.core.TraceAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.IconAnnotator">IconAnnotator</a></h2>
|
||||
<h2><a href="#supervision.annotators.core.CropAnnotator">CropAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.annotators.core.IconAnnotator
|
||||
:::supervision.annotators.core.CropAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.BackgroundOverlayAnnotator">BackgroundOverlayAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.annotators.core.BackgroundOverlayAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.ColorLookup">ColorLookup</a></h2>
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Double Detection Filter
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.overlap_filter.OverlapFilter">OverlapFilter</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.overlap_filter.OverlapFilter
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.overlap_filter.box_non_max_suppression">box_non_max_suppression</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.overlap_filter.box_non_max_suppression
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.overlap_filter.mask_non_max_suppression">mask_non_max_suppression</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.overlap_filter.mask_non_max_suppression
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.overlap_filter.box_non_max_merge">box_non_max_merge</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.overlap_filter.box_non_max_merge
|
||||
@@ -0,0 +1,17 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Metrics
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.metrics.detection.ConfusionMatrix">ConfusionMatrix</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.metrics.detection.ConfusionMatrix
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.metrics.detection.MeanAveragePrecision">MeanAveragePrecision</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.metrics.detection.MeanAveragePrecision
|
||||
@@ -1,6 +1,5 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
# Save Detections
|
||||
@@ -10,3 +9,9 @@ status: new
|
||||
</div>
|
||||
|
||||
:::supervision.detection.tools.csv_sink.CSVSink
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>JSON Sink</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.tools.json_sink.JSONSink
|
||||
+52
-21
@@ -1,71 +1,102 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
# Detection Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>box_iou_batch</h2>
|
||||
<h2><a href="#supervision.detection.utils.box_iou_batch">box_iou_batch</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.box_iou_batch
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>mask_iou_batch</h2>
|
||||
<h2><a href="#supervision.detection.utils.mask_iou_batch">mask_iou_batch</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.mask_iou_batch
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>box_non_max_suppression</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.box_non_max_suppression
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>mask_non_max_suppression</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.mask_non_max_suppression
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>polygon_to_mask</h2>
|
||||
<h2><a href="#supervision.detection.utils.polygon_to_mask">polygon_to_mask</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.polygon_to_mask
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>mask_to_xyxy</h2>
|
||||
<h2><a href="#supervision.detection.utils.mask_to_xyxy">mask_to_xyxy</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.mask_to_xyxy
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>mask_to_polygons</h2>
|
||||
<h2><a href="#supervision.detection.utils.mask_to_polygons">mask_to_polygons</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.mask_to_polygons
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>polygon_to_xyxy</h2>
|
||||
<h2><a href="#supervision.detection.utils.polygon_to_xyxy">polygon_to_xyxy</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.polygon_to_xyxy
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>filter_polygons_by_area</h2>
|
||||
<h2><a href="#supervision.detection.utils.filter_polygons_by_area">filter_polygons_by_area</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.filter_polygons_by_area
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>move_boxes</h2>
|
||||
<h2><a href="#supervision.detection.utils.move_boxes">move_boxes</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.move_boxes
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>scale_boxes</h2>
|
||||
<h2><a href="#supervision.detection.utils.move_masks">move_masks</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.move_masks
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.scale_boxes">scale_boxes</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.scale_boxes
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.clip_boxes">clip_boxes</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.clip_boxes
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.pad_boxes">pad_boxes</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.pad_boxes
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.contains_holes">contains_holes</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.xywh_to_xyxy
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.xywh_to_xyxy">xywh_to_xyxy</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.xcycwh_to_xyxy
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.xcycwh_to_xyxy">xcycwh_to_xyxy</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.contains_holes
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.detection.utils.contains_multiple_segments">contains_multiple_segments</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.detection.utils.contains_multiple_segments
|
||||
@@ -1,13 +0,0 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Color
|
||||
|
||||
:::supervision.draw.color.Color
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>ColorPalette</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.color.ColorPalette
|
||||
@@ -1,53 +0,0 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Draw Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_line</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_line
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_rectangle</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_rectangle
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_filled_rectangle</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_filled_rectangle
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_polygon</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_polygon
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_text</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_text
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>draw_image</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>calculate_dynamic_font_scale</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.calculate_dynamic_text_scale
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>calculate_dynamic_line_thickness</h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.calculate_dynamic_line_thickness
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Position
|
||||
|
||||
:::supervision.geometry.core.Position
|
||||
@@ -4,17 +4,31 @@ comments: true
|
||||
|
||||
# Detect and Annotate
|
||||
|
||||
Supervision offers a streamlined solution to effortlessly annotate predictions from a
|
||||
range of object detection and segmentation models. This guide demonstrates how to
|
||||
execute inference using the YOLOv8 model with either the
|
||||
[Inference](https://github.com/roboflow/inference) or
|
||||
[Ultralytics](https://github.com/ultralytics/ultralytics) packages. Following this,
|
||||
you'll learn how to import these predictions into Supervision for image annotation
|
||||
purposes.
|
||||
Supervision provides a seamless process for annotating predictions generated by various
|
||||
object detection and segmentation models. This guide shows how to perform inference
|
||||
with the [Inference](https://github.com/roboflow/inference),
|
||||
[Ultralytics](https://github.com/ultralytics/ultralytics) or
|
||||
[Transformers](https://github.com/huggingface/transformers) packages. Following this,
|
||||
you'll learn how to import these predictions into Supervision and use them to annotate
|
||||
source image.
|
||||
|
||||
## Run Inference
|
||||

|
||||
|
||||
First, you'll need to obtain predictions from your object detection or segmentation model.
|
||||
## Run Detection
|
||||
|
||||
First, you'll need to obtain predictions from your object detection or segmentation
|
||||
model.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
@@ -23,123 +37,374 @@ First, you'll need to obtain predictions from your object detection or segmentat
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
```
|
||||
|
||||
=== "Inference"
|
||||
=== "Transformers"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
from inference.models.utils import get_roboflow_model
|
||||
import torch
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>)
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
results = model.infer(image)[0]
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
```
|
||||
|
||||
## Load Predictions into Supervision
|
||||
|
||||
Now that we have predictions from a model, we can load them into Supervision.
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
We can do so using the [`sv.Detections.from_ultralytics`](detection/core/#supervision.detection.core.Detections.from_ultralytics) method, which accepts model results from both detection and segmentation models.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
```
|
||||
|
||||
=== "Inference"
|
||||
|
||||
We can do so using the [`sv.Detections.from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) method, which accepts model results from both detection and segmentation models.
|
||||
|
||||
```python
|
||||
```{ .py hl_lines="2 8" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference.models.utils import get_roboflow_model
|
||||
from inference import get_model
|
||||
|
||||
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
```
|
||||
|
||||
You can conveniently load predictions from other computer vision frameworks and libraries using:
|
||||
|
||||
- [`from_deepsparse`](detection/core/#supervision.detection.core.Detections.from_deepsparse) ([Deepsparse](https://github.com/neuralmagic/deepsparse))
|
||||
- [`from_detectron2`](detection/core/#supervision.detection.core.Detections.from_detectron2) ([Detectron2](https://github.com/facebookresearch/detectron2))
|
||||
- [`from_mmdetection`](detection/core/#supervision.detection.core.Detections.from_mmdetection) ([MMDetection](https://github.com/open-mmlab/mmdetection))
|
||||
- [`from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) ([Roboflow Inference](https://github.com/roboflow/inference))
|
||||
- [`from_sam`](detection/core/#supervision.detection.core.Detections.from_sam) ([Segment Anything Model](https://github.com/facebookresearch/segment-anything))
|
||||
- [`from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) ([HuggingFace Transformers](https://github.com/huggingface/transformers))
|
||||
- [`from_yolo_nas`](detection/core/#supervision.detection.core.Detections.from_yolo_nas) ([YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md))
|
||||
|
||||
## Annotate Image
|
||||
|
||||
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](annotators/#supervision.annotators.core.LabelAnnotator) classes. If you are running the segmentation model [`sv.MaskAnnotator`](annotators/#supervision.annotators.core.MaskAnnotator) is a drop-in replacement for [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) that will allow you to draw masks instead of boxes.
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```python
|
||||
We can do so using the [`sv.Detections.from_ultralytics`](detection/core/#supervision.detection.core.Detections.from_ultralytics) method, which accepts model results from both detection and segmentation models.
|
||||
|
||||
```{ .py hl_lines="2 8" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
bounding_box_annotator = sv.BoundingBoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
model.model.names[class_id]
|
||||
for class_id
|
||||
in detections.class_id
|
||||
]
|
||||
|
||||
annotated_image = bounding_box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
We can do so using the [`sv.Detections.from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) method, which accepts model results from both detection and segmentation models.
|
||||
|
||||
```{ .py hl_lines="2 19-21" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(
|
||||
transformers_results=results,
|
||||
id2label=model.config.id2label)
|
||||
```
|
||||
|
||||
You can load predictions from other computer vision frameworks and libraries using:
|
||||
|
||||
- [`from_deepsparse`](/latest/detection/core/#supervision.detection.core.Detections.from_deepsparse) ([Deepsparse](https://github.com/neuralmagic/deepsparse))
|
||||
- [`from_detectron2`](/latest/detection/core/#supervision.detection.core.Detections.from_detectron2) ([Detectron2](https://github.com/facebookresearch/detectron2))
|
||||
- [`from_mmdetection`](/latest/detection/core/#supervision.detection.core.Detections.from_mmdetection) ([MMDetection](https://github.com/open-mmlab/mmdetection))
|
||||
- [`from_sam`](/latest/detection/core/#supervision.detection.core.Detections.from_sam) ([Segment Anything Model](https://github.com/facebookresearch/segment-anything))
|
||||
- [`from_yolo_nas`](/latest/detection/core/#supervision.detection.core.Detections.from_yolo_nas) ([YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md))
|
||||
|
||||
## Annotate Image with Detections
|
||||
|
||||
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoxAnnotator`](/latest/annotators/#supervision.annotators.core.BoxAnnotator) and [`sv.LabelAnnotator`](/latest/annotators/#supervision.annotators.core.LabelAnnotator) classes.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```python
|
||||
```{ .py hl_lines="10-16" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference.models.utils import get_roboflow_model
|
||||
from inference import get_model
|
||||
|
||||
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
|
||||
image = cv2.imread(<PATH TO IMAGE>)
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
bounding_box_annotator = sv.BoundingBoxAnnotator()
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = bounding_box_annotator.annotate(
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||

|
||||
=== "Ultralytics"
|
||||
|
||||
## Display Annotated Image
|
||||
```{ .py hl_lines="10-16" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
To display the annotated image in Jupyter Notebook or Google Colab, use the [`sv.plot_image`](utils/notebook/#supervision.utils.notebook.plot_image) function.
|
||||
model = YOLO("yolov8n.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
```python
|
||||
sv.plot_image(annotated_image)
|
||||
```
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="23-30" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(
|
||||
transformers_results=results,
|
||||
id2label=model.config.id2label)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Display Custom Labels
|
||||
|
||||
By default, [`sv.LabelAnnotator`](/latest/annotators/#supervision.annotators.core.LabelAnnotator)
|
||||
will label each detection with its `class_name` (if possible) or `class_id`. You can
|
||||
override this behavior by passing a list of custom `labels` to the `annotate` method.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="13-17 22" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="13-17 22" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="26-30 35" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(
|
||||
transformers_results=results,
|
||||
id2label=model.config.id2label)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Annotate Image with Segmentations
|
||||
|
||||
If you are running the segmentation model
|
||||
[`sv.MaskAnnotator`](/latest/annotators/#supervision.annotators.core.MaskAnnotator)
|
||||
is a drop-in replacement for
|
||||
[`sv.BoxAnnotator`](/latest/annotators/#supervision.annotators.core.BoxAnnotator)
|
||||
that will allow you to draw masks instead of boxes.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-seg-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n-seg.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```python
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForSegmentation
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50-panoptic")
|
||||
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_segmentation(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(
|
||||
transformers_results=results,
|
||||
id2label=model.config.id2label)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator(text_position=sv.Position.CENTER_OF_MASS)
|
||||
|
||||
labels = [
|
||||
f"{class_name} {confidence:.2f}"
|
||||
for class_name, confidence
|
||||
in zip(detections['class_name'], detections.confidence)
|
||||
]
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||

|
||||
@@ -0,0 +1,325 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Detect Small Objects
|
||||
|
||||
This guide shows how to detect small objects
|
||||
with the [Inference](https://github.com/roboflow/inference),
|
||||
[Ultralytics](https://github.com/ultralytics/ultralytics) or
|
||||
[Transformers](https://github.com/huggingface/transformers) packages using
|
||||
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer).
|
||||
|
||||
<video controls>
|
||||
<source src="https://media.roboflow.com/supervision_detect_small_objects_example.mp4" type="video/mp4">
|
||||
</video>
|
||||
|
||||
## Baseline Detection
|
||||
|
||||
Small object detection in high-resolution images presents challenges due to the objects'
|
||||
size relative to the image resolution.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8x-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8x.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```python
|
||||
import torch
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForSegmentation
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = Image.open(<SOURCE_IMAGE_PATH>)
|
||||
inputs = processor(images=image, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image_slice.size
|
||||
target_size = torch.tensor([[width, height]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
model.config.id2label[class_id]
|
||||
for class_id
|
||||
in detections.class_id
|
||||
]
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Input Resolution
|
||||
|
||||
Modifying the input resolution of images before detection can enhance small object
|
||||
identification at the cost of processing speed and increased memory usage. This method
|
||||
is less effective for ultra-high-resolution images (4K and above).
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="5" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8x-1280")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="7" }
|
||||
import cv2
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8x.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
results = model(image, imgsz=1280)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Inference Slicer
|
||||
|
||||
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer)
|
||||
processes high-resolution images by dividing them into smaller segments, detecting
|
||||
objects within each, and aggregating the results.
|
||||
|
||||
<video controls>
|
||||
<source src="https://media.roboflow.com/supervision_detect_small_objects_example_2.mp4" type="video/mp4">
|
||||
</video>
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="9-14" }
|
||||
import cv2
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8x-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
results = model.infer(image_slice)[0]
|
||||
return sv.Detections.from_inference(results)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="9-14" }
|
||||
import cv2
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8x.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
result = model(image_slice)[0]
|
||||
return sv.Detections.from_ultralytics(result)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="13-28" }
|
||||
import cv2
|
||||
import torch
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from PIL import Image
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
image_slice = cv2.cvtColor(image_slice, cv2.COLOR_BGR2RGB)
|
||||
image_slice = Image.fromarray(image_slice)
|
||||
inputs = processor(images=image_slice, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = image_slice.size
|
||||
target_size = torch.tensor([[width, height]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
return sv.Detections.from_transformers(results)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
labels = [
|
||||
model.config.id2label[class_id]
|
||||
for class_id
|
||||
in detections.class_id
|
||||
]
|
||||
|
||||
annotated_image = box_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections, labels=labels)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Small Object Segmentation
|
||||
|
||||
[`InferenceSlicer`](/latest/detection/tools/inference_slicer/#supervision.detection.tools.inference_slicer.InferenceSlicer) can perform segmentation tasks too.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="6 16 19-20" }
|
||||
import cv2
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8x-seg-640")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
results = model.infer(image_slice)[0]
|
||||
return sv.Detections.from_inference(results)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="6 16 19-20" }
|
||||
import cv2
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8x-seg.pt")
|
||||
image = cv2.imread(<SOURCE_IMAGE_PATH>)
|
||||
|
||||
def callback(image_slice: np.ndarray) -> sv.Detections:
|
||||
result = model(image_slice)[0]
|
||||
return sv.Detections.from_ultralytics(result)
|
||||
|
||||
slicer = sv.InferenceSlicer(callback = callback)
|
||||
detections = slicer(image)
|
||||
|
||||
mask_annotator = sv.MaskAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_image = mask_annotator.annotate(
|
||||
scene=image, detections=detections)
|
||||
annotated_image = label_annotator.annotate(
|
||||
scene=annotated_image, detections=detections)
|
||||
```
|
||||
|
||||

|
||||
@@ -0,0 +1,455 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
With Supervision, you can load and manipulate classification, object detection, and
|
||||
segmentation datasets. This tutorial will walk you through how to load, split, merge,
|
||||
visualize, and augment datasets in Supervision.
|
||||
|
||||
## Download Dataset
|
||||
|
||||
In this tutorial, we will use a dataset from
|
||||
[Roboflow Universe](https://universe.roboflow.com/), a public repository of
|
||||
thousands of computer vision datasets. If you already have your dataset in
|
||||
[COCO](https://roboflow.com/formats/coco-json),
|
||||
[YOLO](https://roboflow.com/formats/yolov8-pytorch-txt),
|
||||
or [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format, you can skip this
|
||||
section.
|
||||
|
||||
```bash
|
||||
pip install roboflow
|
||||
```
|
||||
|
||||
Next, log into your Roboflow account and download the dataset of your choice in the
|
||||
COCO, YOLO, or Pascal VOC format. You can customize the following code snippet with
|
||||
your workspace ID, project ID, and version number.
|
||||
|
||||
=== "COCO"
|
||||
|
||||
```python
|
||||
import roboflow
|
||||
|
||||
roboflow.login()
|
||||
|
||||
rf = roboflow.Roboflow()
|
||||
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
|
||||
dataset = project.version('<PROJECT_VERSION>').download("coco")
|
||||
```
|
||||
|
||||
=== "YOLO"
|
||||
|
||||
```python
|
||||
import roboflow
|
||||
|
||||
roboflow.login()
|
||||
|
||||
rf = roboflow.Roboflow()
|
||||
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
|
||||
dataset = project.version('<PROJECT_VERSION>').download("yolov8")
|
||||
```
|
||||
|
||||
=== "Pascal VOC"
|
||||
|
||||
```python
|
||||
import roboflow
|
||||
|
||||
roboflow.login()
|
||||
|
||||
rf = roboflow.Roboflow()
|
||||
project = rf.workspace('<WORKSPACE_ID>').project('<PROJECT_ID>')
|
||||
dataset = project.version('<PROJECT_VERSION>').download("voc")
|
||||
```
|
||||
|
||||
## Load Dataset
|
||||
|
||||
The Supervision library provides convenient functions to load datasets in various
|
||||
formats. If your dataset is already split into train, test, and valid subsets, you can
|
||||
load each of those as separate [`sv.DetectionDataset`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset)
|
||||
instances.
|
||||
|
||||
=== "COCO"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.from_coco`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_coco) to load annotations in [COCO](https://roboflow.com/formats/coco-json) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/train',
|
||||
annotations_path=f'{dataset.location}/train/_annotations.coco.json',
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/valid',
|
||||
annotations_path=f'{dataset.location}/valid/_annotations.coco.json',
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/test',
|
||||
annotations_path=f'{dataset.location}/test/_annotations.coco.json',
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
```
|
||||
|
||||
=== "YOLO"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.from_yolo`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_yolo) to load annotations in [YOLO](https://roboflow.com/formats/yolov8-pytorch-txt) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/train/images',
|
||||
annotations_directory_path=f'{dataset.location}/train/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/valid/images',
|
||||
annotations_directory_path=f'{dataset.location}/valid/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/test/images',
|
||||
annotations_directory_path=f'{dataset.location}/test/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
```
|
||||
|
||||
=== "Pascal VOC"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.from_pascal_voc`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.from_pascal_voc) to load annotations in [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/train/images',
|
||||
annotations_directory_path=f'{dataset.location}/train/labels'
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/valid/images',
|
||||
annotations_directory_path=f'{dataset.location}/valid/labels'
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/test/images',
|
||||
annotations_directory_path=f'{dataset.location}/test/labels'
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
```
|
||||
|
||||
## Split Dataset
|
||||
|
||||
If your dataset is not already split into train, test, and valid subsets, you can
|
||||
easily do so using the [`sv.DetectionDataset.split`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.split)
|
||||
method. We can split it as follows, ensuring a random shuffle of the data.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
len(ds)
|
||||
# 1000
|
||||
|
||||
ds_train, ds = ds.split(split_ratio=0.8, shuffle=True)
|
||||
ds_valid, ds_test = ds.split(split_ratio=0.5, shuffle=True)
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
```
|
||||
|
||||
## Merge Dataset
|
||||
|
||||
If you have multiple datasets that you would like to merge, you can do so using the
|
||||
[`sv.DetectionDataset.merge`](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.merge)
|
||||
method.
|
||||
|
||||
=== "COCO"
|
||||
|
||||
```{ .py hl_lines="22-28" }
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/train',
|
||||
annotations_path=f'{dataset.location}/train/_annotations.coco.json',
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/valid',
|
||||
annotations_path=f'{dataset.location}/valid/_annotations.coco.json',
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_coco(
|
||||
images_directory_path=f'{dataset.location}/test',
|
||||
annotations_path=f'{dataset.location}/test/_annotations.coco.json',
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
|
||||
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
|
||||
|
||||
ds.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds)
|
||||
# 1000
|
||||
```
|
||||
|
||||
=== "YOLO"
|
||||
|
||||
```{ .py hl_lines="25-31" }
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/train/images',
|
||||
annotations_directory_path=f'{dataset.location}/train/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/valid/images',
|
||||
annotations_directory_path=f'{dataset.location}/valid/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_yolo(
|
||||
images_directory_path=f'{dataset.location}/test/images',
|
||||
annotations_directory_path=f'{dataset.location}/test/labels',
|
||||
data_yaml_path=f'{dataset.location}/data.yaml'
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
|
||||
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
|
||||
|
||||
ds.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds)
|
||||
# 1000
|
||||
```
|
||||
|
||||
=== "Pascal VOC"
|
||||
|
||||
```{ .py hl_lines="22-28" }
|
||||
import supervision as sv
|
||||
|
||||
ds_train = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/train/images',
|
||||
annotations_directory_path=f'{dataset.location}/train/labels'
|
||||
)
|
||||
ds_valid = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/valid/images',
|
||||
annotations_directory_path=f'{dataset.location}/valid/labels'
|
||||
)
|
||||
ds_test = sv.DetectionDataset.from_pascal_voc(
|
||||
images_directory_path=f'{dataset.location}/test/images',
|
||||
annotations_directory_path=f'{dataset.location}/test/labels'
|
||||
)
|
||||
|
||||
ds_train.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds_train), len(ds_valid), len(ds_test)
|
||||
# 800, 100, 100
|
||||
|
||||
ds = sv.DetectionDataset.merge([ds_train, ds_valid, ds_test])
|
||||
|
||||
ds.classes
|
||||
# ['person', 'bicycle', 'car', ...]
|
||||
|
||||
len(ds)
|
||||
# 1000
|
||||
```
|
||||
|
||||
## Iterate over Dataset
|
||||
|
||||
There are two ways to loop over a `sv.DetectionDataset`: using a direct
|
||||
[for loop](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.__iter__)
|
||||
called on the `sv.DetectionDataset` instance or loading `sv.DetectionDataset` entries
|
||||
[by index](https://supervision.roboflow.com/latest/datasets/core/#supervision.dataset.core.DetectionDataset.__getitem__).
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
# Option 1
|
||||
for image_path, image, annotations in ds:
|
||||
... # Process each image and its annotations
|
||||
|
||||
# Option 2
|
||||
for idx in range(len(ds)):
|
||||
image_path, image, annotations = ds[idx]
|
||||
... # Process the image and annotations at index `idx`
|
||||
```
|
||||
|
||||
## Visualize Dataset
|
||||
|
||||
The Supervision library provides tools for easily visualizing your detection dataset.
|
||||
You can create a grid of annotated images to quickly inspect your data and labels.
|
||||
First, initialize the [`sv.BoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoxAnnotator)
|
||||
and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator).
|
||||
Then, iterate through a subset of the dataset (e.g., the first 25 images), drawing
|
||||
bounding boxes and class labels on each image. Finally, combine the annotated images
|
||||
into a grid for display.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
box_annotator = sv.BoxAnnotator()
|
||||
label_annotator = sv.LabelAnnotator()
|
||||
|
||||
annotated_images = []
|
||||
for i in range(16):
|
||||
_, image, annotations = ds[i]
|
||||
|
||||
labels = [ds.classes[class_id] for class_id in annotations.class_id]
|
||||
|
||||
annotated_image = image.copy()
|
||||
annotated_image = box_annotator.annotate(annotated_image, annotations)
|
||||
annotated_image = label_annotator.annotate(annotated_image, annotations, labels)
|
||||
annotated_images.append(annotated_image)
|
||||
|
||||
grid = sv.create_tiles(
|
||||
annotated_images,
|
||||
grid_size=(4, 4),
|
||||
single_tile_size=(400, 400),
|
||||
tile_padding_color=sv.Color.WHITE,
|
||||
tile_margin_color=sv.Color.WHITE
|
||||
)
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Save Dataset
|
||||
|
||||
=== "COCO"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.as_coco`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_coco) method to save annotations in [COCO](https://roboflow.com/formats/coco-json) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
ds.as_coco(
|
||||
images_directory_path='<IMAGE_DIRECTORY_PATH>',
|
||||
annotations_path='<ANNOTATIONS_PATH>'
|
||||
)
|
||||
```
|
||||
|
||||
=== "YOLO"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.as_yolo`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_yolo) method to save annotations in [YOLO](https://roboflow.com/formats/yolov8-pytorch-txt) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
ds.as_yolo(
|
||||
images_directory_path='<IMAGE_DIRECTORY_PATH>',
|
||||
annotations_directory_path='<ANNOTATIONS_DIRECTORY_PATH>',
|
||||
data_yaml_path='<DATA_YAML_PATH>'
|
||||
)
|
||||
```
|
||||
|
||||
=== "Pascal VOC"
|
||||
|
||||
We can do so using the [`sv.DetectionDataset.as_pascal_voc`](https://supervision.roboflow.com/datasets/#supervision.dataset.core.DetectionDataset.as_pascal_voc) method to save annotations in [Pascal VOC](https://roboflow.com/formats/pascal-voc-xml) format.
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
ds.as_pascal_voc(
|
||||
images_directory_path='<IMAGE_DIRECTORY_PATH>',
|
||||
annotations_directory_path='<ANNOTATIONS_DIRECTORY_PATH>'
|
||||
)
|
||||
```
|
||||
|
||||
## Augment Dataset
|
||||
|
||||
In this section, we'll explore using Supervision in combination with Albumentations to
|
||||
augment our dataset. Data augmentation is a common technique in computer vision to
|
||||
increase the size and diversity of training datasets, leading to improved model
|
||||
performance and generalization.
|
||||
|
||||
```bash
|
||||
pip install augmentation
|
||||
```
|
||||
|
||||
Albumentations provides a flexible and powerful API for image augmentation. The core of
|
||||
the library is the [`Compose`](https://albumentations.ai/docs/api_reference/full_reference/?h=compose#albumentations.core.composition.Compose)
|
||||
class, which allows you to chain multiple image transformations together. Each
|
||||
transformation is defined using a dedicated class, such as
|
||||
[`HorizontalFlip`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.geometric.transforms.HorizontalFlip),
|
||||
[`RandomBrightnessContrast`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.transforms.RandomBrightnessContrast),
|
||||
or [`Perspective`](https://albumentations.ai/docs/api_reference/full_reference/?h=horizontalflip#albumentations.augmentations.geometric.transforms.Perspective).
|
||||
|
||||
```python
|
||||
import albumentations as A
|
||||
|
||||
augmentation = A.Compose(
|
||||
transforms=[
|
||||
A.Perspective(p=0.1),
|
||||
A.HorizontalFlip(p=0.5),
|
||||
A.RandomBrightnessContrast(p=0.5)
|
||||
],
|
||||
bbox_params=A.BboxParams(
|
||||
format='pascal_voc',
|
||||
label_fields=['category']
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
The key is to set `format='pascal_voc'`, which corresponds to the
|
||||
`[x_min, y_min, x_max, y_max]` bounding box format used in Supervision.
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
import supervision as sv
|
||||
from dataclasses import replace
|
||||
|
||||
ds = sv.DetectionDataset(...)
|
||||
|
||||
_, original_image, original_annotations = ds[0]
|
||||
|
||||
output = augmentation(
|
||||
image=original_image,
|
||||
bboxes=original_annotations.xyxy,
|
||||
category=original_annotations.class_id
|
||||
)
|
||||
|
||||
augmented_image = output['image']
|
||||
augmented_annotations = replace(
|
||||
original_annotations,
|
||||
xyxy=np.array(output['bboxes']),
|
||||
class_id=np.array(output['category'])
|
||||
)
|
||||
```
|
||||
|
||||

|
||||
@@ -0,0 +1,298 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Save Detections
|
||||
|
||||
Supervision enables an easy way to save detections in .CSV and .JSON files for offline
|
||||
processing. This guide demonstrates how to perform video inference using the
|
||||
[Inference](https://github.com/roboflow/inference),
|
||||
[Ultralytics](https://github.com/ultralytics/ultralytics) or
|
||||
[Transformers](https://github.com/huggingface/transformers) packages and save their results with
|
||||
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) and
|
||||
[`sv.JSONSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink).
|
||||
|
||||
## Run Detection
|
||||
|
||||
First, you'll need to obtain predictions from your object detection or segmentation
|
||||
model. You can learn more on this topic in our
|
||||
[How to Detect and Annotate](/latest/how_to/detect_and_annotate.md) guide.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
for frame in frames_generator:
|
||||
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
for frame in frames_generator:
|
||||
|
||||
results = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```python
|
||||
import torch
|
||||
import supervision as sv
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
for frame in frames_generator:
|
||||
|
||||
frame = sv.cv2_to_pillow(frame)
|
||||
inputs = processor(images=frame, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = frame.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results)
|
||||
```
|
||||
|
||||
## Save Detections as CSV
|
||||
|
||||
To save detections to a `.CSV` file, open our
|
||||
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
|
||||
and then pass the
|
||||
[`sv.Detections`](/latest/detection/core/#supervision.detection.core.Detections)
|
||||
object resulting from the inference to it. Its fields are parsed and saved on disk.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="7 12" }
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame in frames_generator:
|
||||
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
sink.append(detections, {})
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="7 12" }
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame in frames_generator:
|
||||
|
||||
results = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
sink.append(detections, {})
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="9 23" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame in frames_generator:
|
||||
|
||||
frame = sv.cv2_to_pillow(frame)
|
||||
inputs = processor(images=frame, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = frame.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results)
|
||||
sink.append(detections, {})
|
||||
```
|
||||
|
||||
| x_min | y_min | x_max | y_max | class_id | confidence | tracker_id | class_name |
|
||||
|---------|----------|---------|----------|----------|------------|------------|------------|
|
||||
| 2941.14 | 1269.31 | 3220.77 | 1500.67 | 2 | 0.8517 | | car |
|
||||
| 944.889 | 899.641 | 1235.42 | 1308.80 | 7 | 0.6752 | | truck |
|
||||
| 1439.78 | 1077.79 | 1621.27 | 1231.40 | 2 | 0.6450 | | car |
|
||||
|
||||
## Custom Fields
|
||||
|
||||
Besides regular fields in
|
||||
[`sv.Detections`](/latest/detection/core/#supervision.detection.core.Detections),
|
||||
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
|
||||
also allows you to add custom information to each row, which can be passed via the
|
||||
`custom_data` dictionary. Let's utilize this feature to save information about the
|
||||
frame index from which the detections originate.
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="8 12" }
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="8 12" }
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
results = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="10 23" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.CSVSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
frame = sv.cv2_to_pillow(frame)
|
||||
inputs = processor(images=frame, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = frame.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
|
||||
| x_min | y_min | x_max | y_max | class_id | confidence | tracker_id | class_name | frame_index |
|
||||
|---------|----------|---------|----------|----------|------------|------------|------------|-------------|
|
||||
| 2941.14 | 1269.31 | 3220.77 | 1500.67 | 2 | 0.8517 | | car | 0 |
|
||||
| 944.889 | 899.641 | 1235.42 | 1308.80 | 7 | 0.6752 | | truck | 0 |
|
||||
| 1439.78 | 1077.79 | 1621.27 | 1231.40 | 2 | 0.6450 | | car | 0 |
|
||||
|
||||
## Save Detections as JSON
|
||||
|
||||
If you prefer to save the result in a `.JSON` file instead of a `.CSV` file, all you
|
||||
need to do is replace
|
||||
[`sv.CSVSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink)
|
||||
with
|
||||
[`sv.JSONSink`](/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.JSONSink).
|
||||
|
||||
=== "Inference"
|
||||
|
||||
```{ .py hl_lines="7" }
|
||||
import supervision as sv
|
||||
from inference import get_model
|
||||
|
||||
model = get_model(model_id="yolov8n-640")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
results = model.infer(image)[0]
|
||||
detections = sv.Detections.from_inference(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
|
||||
=== "Ultralytics"
|
||||
|
||||
```{ .py hl_lines="7" }
|
||||
import supervision as sv
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolov8n.pt")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
results = model(frame)[0]
|
||||
detections = sv.Detections.from_ultralytics(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
|
||||
=== "Transformers"
|
||||
|
||||
```{ .py hl_lines="9" }
|
||||
import torch
|
||||
import supervision as sv
|
||||
from transformers import DetrImageProcessor, DetrForObjectDetection
|
||||
|
||||
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
|
||||
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
|
||||
frames_generator = sv.get_video_frames_generator(<SOURCE_VIDEO_PATH>)
|
||||
|
||||
with sv.JSONSink(<TARGET_CSV_PATH>) as sink:
|
||||
for frame_index, frame in enumerate(frames_generator):
|
||||
|
||||
frame = sv.cv2_to_pillow(frame)
|
||||
inputs = processor(images=frame, return_tensors="pt")
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model(**inputs)
|
||||
|
||||
width, height = frame.size
|
||||
target_size = torch.tensor([[height, width]])
|
||||
results = processor.post_process_object_detection(
|
||||
outputs=outputs, target_sizes=target_size)[0]
|
||||
detections = sv.Detections.from_transformers(results)
|
||||
sink.append(detections, {"frame_index": frame_index})
|
||||
```
|
||||
+47
-12
@@ -1,19 +1,20 @@
|
||||
---
|
||||
template: index.html
|
||||
comments: true
|
||||
hide:
|
||||
- navigation
|
||||
- toc
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
<p>
|
||||
<a align="center" href="" target="_blank">
|
||||
<img
|
||||
width="850"
|
||||
src="https://media.roboflow.com/open-source/supervision/rf-supervision-banner.png?updatedAt=1678995927529"
|
||||
>
|
||||
</a>
|
||||
</p>
|
||||
<div class="md-typeset">
|
||||
<h1></h1>
|
||||
</div>
|
||||
|
||||
<div align="center" id="logo">
|
||||
<a align="center" href="" target="_blank">
|
||||
<img width="850"
|
||||
src="https://media.roboflow.com/open-source/supervision/rf-supervision-banner.png?updatedAt=1678995927529">
|
||||
</a>
|
||||
</div>
|
||||
|
||||
## 👋 Hello
|
||||
@@ -29,7 +30,7 @@ We write your reusable computer vision tools. Whether you need to load your data
|
||||
|
||||
## 💻 Install
|
||||
|
||||
You can install `supervision` with pip in a
|
||||
You can install `supervision` in a
|
||||
[**Python>=3.8**](https://www.python.org/) environment.
|
||||
|
||||
!!! example "pip install (recommended)"
|
||||
@@ -45,7 +46,25 @@ You can install `supervision` with pip in a
|
||||
If you require the full version of `supervision` with GUI support you can install the desktop version. This version includes the GUI components of OpenCV, allowing you to display images and videos on the screen.
|
||||
|
||||
```bash
|
||||
pip install supervision[desktop]
|
||||
pip install "supervision[desktop]"
|
||||
```
|
||||
|
||||
!!! example "conda/mamba install"
|
||||
|
||||
=== "conda"
|
||||
|
||||
[](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision)
|
||||
|
||||
```bash
|
||||
conda install -c conda-forge supervision
|
||||
```
|
||||
|
||||
=== "mamba"
|
||||
|
||||
[](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision) [](https://anaconda.org/conda-forge/supervision)
|
||||
|
||||
```bash
|
||||
mamba install -c conda-forge supervision
|
||||
```
|
||||
|
||||
!!! example "git clone (for development)"
|
||||
@@ -87,7 +106,6 @@ You can install `supervision` with pip in a
|
||||
poetry install --extras "desktop"
|
||||
```
|
||||
|
||||
|
||||
## 🚀 Quickstart
|
||||
|
||||
<div class="grid cards" markdown>
|
||||
@@ -108,6 +126,14 @@ You can install `supervision` with pip in a
|
||||
|
||||
[:octicons-arrow-right-24: Tutorial](how_to/track_objects.md)
|
||||
|
||||
- __Detect Small Objects__
|
||||
|
||||
---
|
||||
|
||||
Learn how to detect small objects in images
|
||||
|
||||
[:octicons-arrow-right-24: Tutorial](how_to/detect_small_objects.md)
|
||||
|
||||
- > __Count Objects Crossing Line__
|
||||
|
||||
---
|
||||
@@ -120,4 +146,13 @@ You can install `supervision` with pip in a
|
||||
|
||||
Master the techniques to selectively filter and focus on objects within a specific zone
|
||||
|
||||
- **Cheatsheet**
|
||||
|
||||
***
|
||||
|
||||
Access a quick reference guide to the most common `supervision` functions
|
||||
|
||||
[:octicons-arrow-right-24: Cheatsheet](https://roboflow.github.io/cheatsheet-supervision/)
|
||||
|
||||
|
||||
</div>
|
||||
@@ -55,7 +55,7 @@ document.addEventListener("DOMContentLoaded", function () {
|
||||
|
||||
let authorAvatarsHTML = authorDataArray.map((authorData, index) => {
|
||||
const marginLeft = index === 0 ? '0' : '-10px';
|
||||
const zIndex = 100 - index;
|
||||
const zIndex = 4 - index;
|
||||
return `
|
||||
<div
|
||||
class="author-container"
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Annotators
|
||||
|
||||
=== "VertexAnnotator"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
key_points = sv.KeyPoints(...)
|
||||
|
||||
vertex_annotator = sv.VertexAnnotator(
|
||||
color=sv.Color.GREEN,
|
||||
radius=10
|
||||
)
|
||||
annotated_frame = vertex_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
key_points=key_points
|
||||
)
|
||||
```
|
||||
|
||||
<div class="result" markdown>
|
||||
|
||||
{ align=center width="800" }
|
||||
|
||||
</div>
|
||||
|
||||
=== "EdgeAnnotator"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
key_points = sv.KeyPoints(...)
|
||||
|
||||
edge_annotator = sv.EdgeAnnotator(
|
||||
color=sv.Color.GREEN,
|
||||
thickness=5
|
||||
)
|
||||
annotated_frame = edge_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
key_points=key_points
|
||||
)
|
||||
```
|
||||
|
||||
<div class="result" markdown>
|
||||
|
||||
{ align=center width="800" }
|
||||
|
||||
</div>
|
||||
|
||||
=== "VertexLabelAnnotator"
|
||||
|
||||
```python
|
||||
import supervision as sv
|
||||
|
||||
image = ...
|
||||
key_points = sv.KeyPoints(...)
|
||||
|
||||
vertex_label_annotator = sv.VertexLabelAnnotator(
|
||||
color=sv.Color.GREEN,
|
||||
text_color=sv.Color.BLACK,
|
||||
border_radius=5
|
||||
)
|
||||
annotated_frame = vertex_label_annotator.annotate(
|
||||
scene=image.copy(),
|
||||
key_points=key_points
|
||||
)
|
||||
```
|
||||
|
||||
<div class="result" markdown>
|
||||
|
||||
{ align=center width="800" }
|
||||
|
||||
</div>
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.keypoint.annotators.VertexAnnotator">VertexAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.keypoint.annotators.VertexAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.keypoint.annotators.EdgeAnnotator">EdgeAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.keypoint.annotators.EdgeAnnotator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.keypoint.annotators.VertexLabelAnnotator">VertexLabelAnnotator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.keypoint.annotators.VertexLabelAnnotator
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
# Keypoint Detection
|
||||
|
||||
:::supervision.keypoint.core.KeyPoints
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Detection Metrics
|
||||
|
||||
!!! warning
|
||||
|
||||
Evaluation API is still fluid and may change. If you use Evaluation API in your project until further notice, freeze the
|
||||
`supervision` version in your `requirements.txt` or `setup.py`.
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.metrics.detection.ConfusionMatrix">ConfusionMatrix</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.metrics.detection.ConfusionMatrix
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.annotators.core.MeanAveragePrecision">MeanAveragePrecision</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.metrics.detection.MeanAveragePrecision
|
||||
+277
-82
@@ -2,7 +2,9 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "TfUZYrKbqlF7"
|
||||
},
|
||||
"source": [
|
||||
"# Annotate Video with Detections\n",
|
||||
"\n",
|
||||
@@ -10,9 +12,72 @@
|
||||
"\n",
|
||||
"[](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/annotate-video-with-detections.ipynb)\n",
|
||||
"\n",
|
||||
"One of the most common requirements of computer vision applications is detecting objects in images and displaying bounding boxes around those objects. In this cookbook we'll walk through the steps on how to utilize the open source Roboflow ecosystem to accomplish this task on a video. Let's dive in! \n",
|
||||
"One of the most common requirements of computer vision applications is detecting objects in images and displaying bounding boxes around those objects. In this cookbook we'll walk through the steps on how to utilize the open source Roboflow ecosystem to accomplish this task on a video. Let's dive in!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "rUo1u17yqlF9"
|
||||
},
|
||||
"source": [
|
||||
"## Before you start\n",
|
||||
"\n",
|
||||
"## Installing Dependencies \n",
|
||||
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "0Er370WTqlF9",
|
||||
"outputId": "57b00a1e-87ea-4912-b4f9-5b94f0e6bf48",
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Fri Feb 23 03:15:00 2024 \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
|
||||
"|-----------------------------------------+----------------------+----------------------+\n",
|
||||
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
|
||||
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
|
||||
"| | | MIG M. |\n",
|
||||
"|=========================================+======================+======================|\n",
|
||||
"| 0 Tesla V100-SXM2-16GB Off | 00000000:00:04.0 Off | 0 |\n",
|
||||
"| N/A 33C P0 24W / 300W | 0MiB / 16384MiB | 0% Default |\n",
|
||||
"| | | N/A |\n",
|
||||
"+-----------------------------------------+----------------------+----------------------+\n",
|
||||
" \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| Processes: |\n",
|
||||
"| GPU GI CI PID Type Process name GPU Memory |\n",
|
||||
"| ID ID Usage |\n",
|
||||
"|=======================================================================================|\n",
|
||||
"| No running processes found |\n",
|
||||
"+---------------------------------------------------------------------------------------+\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!nvidia-smi"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "WuSnN7ejqlF-"
|
||||
},
|
||||
"source": [
|
||||
"## Installing Dependencies\n",
|
||||
"\n",
|
||||
"In this cookbook we'll be utilizing the open source packages [Inference](https://inference.roboflow.com/) and [Supervision](https://supervision.roboflow.com/latest/) to accomplish our goals. Let's get those installed in our notebook with pip."
|
||||
]
|
||||
@@ -21,40 +86,68 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "1YgQGum4qlF-",
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"!pip install -q inference \"supervision[assets]\""
|
||||
"!pip install -q inference-gpu \"supervision[assets]\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "PZRnzh7EqlF-"
|
||||
},
|
||||
"source": [
|
||||
"## Download a Video Asset\n",
|
||||
"\n",
|
||||
"First, let's download a video that we can detect objects in. Supervision comes with a great utility called Assets to help us hit the ground running. Wehn we run this script, the video is saved in our local directory and can be accessed with the variable `path_to_video`."
|
||||
"First, let's download a video that we can detect objects in. Supervision comes with a great utility called Assets to help us hit the ground running. When we run this script, the video is saved in our local directory and can be accessed with the variable `path_to_video`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "NMg4KOFiqlF-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"\n",
|
||||
"# Download a supervision video asset \n",
|
||||
"# Download a supervision video asset\n",
|
||||
"path_to_video = download_assets(VideoAssets.PEOPLE_WALKING)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "x93oxF0zrQOU"
|
||||
},
|
||||
"source": [
|
||||
"As a result, we've downloaded a video. Let's take a look at the video below. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
|
||||
"\n",
|
||||
"<div style=\"display: flex; justify-content: center;\">\n",
|
||||
" <video controls width=\"1000\"\">\n",
|
||||
" <source\n",
|
||||
" src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\"\n",
|
||||
" type=\"video/mp4\"\n",
|
||||
" >\n",
|
||||
" </video>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "T8bU7fQ6qlF-"
|
||||
},
|
||||
"source": [
|
||||
"## Detecting Objects\n",
|
||||
"\n",
|
||||
@@ -63,128 +156,230 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "hlONWHfQqlF-",
|
||||
"outputId": "81b277fb-ea7c-4242-f49d-5af0c665ea58"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Detections(xyxy=array([[1140., 951., 1245., 1079.],\n",
|
||||
" [ 666., 648., 745., 854.],\n",
|
||||
" [ 34., 794., 142., 990.],\n",
|
||||
" [1140., 505., 1211., 657.],\n",
|
||||
" [ 260., 438., 332., 612.],\n",
|
||||
" [1413., 702., 1523., 887.],\n",
|
||||
" [1462., 472., 1543., 643.],\n",
|
||||
" [1446., 318., 1516., 483.],\n",
|
||||
" [ 753., 451., 821., 623.],\n",
|
||||
" [ 924., 172., 983., 307.],\n",
|
||||
" [1791., 144., 1852., 275.],\n",
|
||||
" [ 93., 132., 146., 251.],\n",
|
||||
" [ 708., 240., 765., 388.],\n",
|
||||
" [ 200., 44., 267., 161.],\n",
|
||||
" [1204., 131., 1255., 266.],\n",
|
||||
" [ 569., 267., 628., 408.],\n",
|
||||
" [1163., 150., 1210., 280.],\n",
|
||||
" [ 799., 78., 847., 204.],\n",
|
||||
" [1690., 152., 1751., 283.],\n",
|
||||
" [ 344., 495., 396., 641.],\n",
|
||||
" [1722., 77., 1782., 178.]]),\n",
|
||||
" mask=None,\n",
|
||||
" confidence=array([0.83215541, 0.80572134, 0.7919845 , 0.7912274 , 0.77121079,\n",
|
||||
" 0.7599591 , 0.75711554, 0.75494027, 0.73076195, 0.71452248,\n",
|
||||
" 0.69572842, 0.65269446, 0.63952065, 0.62914598, 0.61361706,\n",
|
||||
" 0.5968492 , 0.55311316, 0.5470854 , 0.54070991, 0.52209878,\n",
|
||||
" 0.41217673]),\n",
|
||||
" class_id=array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),\n",
|
||||
" tracker_id=None,\n",
|
||||
" data={'class_name': array(['person', 'person', 'person', 'person', 'person', 'person',\n",
|
||||
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
|
||||
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
|
||||
" 'person', 'person', 'person'], dtype='<U6')})\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import supervision as sv\n",
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"from inference.models.utils import get_roboflow_model\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"if __name__ == \"__main__\":\n",
|
||||
" # Download the video asset from Supervision assets.\n",
|
||||
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
|
||||
"# Load a yolov8 model from roboflow.\n",
|
||||
"model = get_roboflow_model(\"yolov8s-640\")\n",
|
||||
"\n",
|
||||
" # Load a yolov8 nano model from roboflow.\n",
|
||||
" model = get_roboflow_model(\"yolov8n-640\")\n",
|
||||
"# Create a frame generator and video info object from supervision utilities.\n",
|
||||
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
|
||||
"\n",
|
||||
" # Create a frame generator and video info object from supervision utilities.\n",
|
||||
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
|
||||
"# Yield a single frame from the generator.\n",
|
||||
"frame = next(frame_generator)\n",
|
||||
"\n",
|
||||
" # Yield a single frame from the generator.\n",
|
||||
" frame = next(frame_generator)\n",
|
||||
"# Run inference on our frame\n",
|
||||
"result = model.infer(frame)[0]\n",
|
||||
"\n",
|
||||
" # Run inference on our frame.\n",
|
||||
" result = model.infer(frame)[0]\n",
|
||||
"# Parse result into detections data model.\n",
|
||||
"detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
" # Parse result into detections data model.\n",
|
||||
" detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
" # Display the detections data model.\n",
|
||||
" print(detections)"
|
||||
"# Pretty Print the resulting detections.\n",
|
||||
"from pprint import pprint\n",
|
||||
"pprint(detections)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "mL7QrQvyqlF-"
|
||||
},
|
||||
"source": [
|
||||
"First, we load our model using the method `get_roboflow_model()`. Notice how we pass in a `model_id`? We're using an [alias](https://inference.roboflow.com/reference_pages/model_aliases/) here. This is where we can pass in other models from Roboflow Universe like this [rock, paper, scissors](https://universe.roboflow.com/roboflow-58fyf/rock-paper-scissors-sxsw) model utilizing our roboflow api key. \n",
|
||||
"First, we load our model using the method `get_roboflow_model()`. Notice how we pass in a `model_id`? We're using an [alias](https://inference.roboflow.com/reference_pages/model_aliases/) here. This is where we can pass in other models from Roboflow Universe like this [rock, paper, scissors](https://universe.roboflow.com/roboflow-58fyf/rock-paper-scissors-sxsw) model utilizing our roboflow api key.\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"model = get_roboflow_mode(\n",
|
||||
" model_id=\"rock-paper-scissors-sxsw/11\", \n",
|
||||
" model_id=\"rock-paper-scissors-sxsw/11\",\n",
|
||||
" api_key=\"roboflow_private_api_key\"\n",
|
||||
")\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"If you don't have an api key, you can [create an free Roboflow account](https://app.roboflow.com/login). This model wouldn't be much help with detecting people, but it's a nice exercise to see how our code becomes model agnostic!\n",
|
||||
"\n",
|
||||
"We then create a `frame_generator` object and yeild a single frame for inference using `next()`. We pass our frame to `model.infer()` to run inference, then pass that data into a little helpfer function called `sv.Detections.from_inference()` to parse it. Lastly we print our detections to show we are in fact detecting a few people in the frame! \n",
|
||||
"We then create a `frame_generator` object and yeild a single frame for inference using `next()`. We pass our frame to `model.infer()` to run inference. After, we pass that data into a little helpfer function called `sv.Detections.from_inference()` to parse it. Lastly we print our detections to show we are in fact detecting a few people in the frame!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "tM6geyUBwdej"
|
||||
},
|
||||
"source": [
|
||||
"## Annotaing the Frame with Bounding Boxes\n",
|
||||
"\n",
|
||||
"## Saving Bounding Boxes to the Video\n",
|
||||
"Now that we're detecting images, let's get to the fun part. Let's annotate the frame and display the bounding boxes on the frame."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "8gkUxKei0RaV",
|
||||
"outputId": "84a95332-70cf-4f26-c4bb-15479afafd04"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9WYxlyXUeCn8Re58hz8mTY2VlZdY8Vw/V7InN7ubUFEnJFEVZlGzYhmRd0LgPF1cGBBgG/OIXPxr478sv3xf5vxANSpYlSpRJUCRFimK3mt3NLrK75p5q6qrKzMrMyso5z7x3/A+xV+wVsWOfc6rlHz8E5CpknXP2EMOKFWuKFSuEUkphF3ZhF3ZhF3ZhF3ZhF3ZhF3ZhF3bhHynI/383YBd2YRd2YRd2YRd2YRd2YRd2YRd24R8Cu4btLuzCLuzCLuzCLuzCLuzCLuzCLvyjhl3Ddhd2YRd2YRd2YRd2YRd2YRd2YRf+UcOuYbsLu7ALu7ALu7ALu7ALu7ALu7AL/6hh17DdhV3YhV3YhV3YhV3YhV3YhV3YhX/UsGvY7sIu7MIu7MIu7MIu7MIu7MIu7MI/atg1bHdhF3ZhF3ZhF3ZhF3ZhF3ZhF3bhHzXsGra7sAu7sAu7sAu7sAu7sAu7sAu78I8adg3bXdiFXdiFXdiFXdiFXdiFXdiFXfhHDeGgD/6n//Sfet4XQlifeaCUst6hPyklCoUCpJSZ+73KiaII3W4XURRBKWWVyd93y6I66VoYhgjDMPOMr14hBJRS5g8ApJT40z/9U1y+fBnPPPMMDh48iImJCQwNDZl6+LtbW1vodrvY2trC/Pw8Zmdn8eSTT6JQKJjnpJQWPqh+Xm8cx5m+PMw48LL4O3TdxSO/F0UR4ji23o/jGJ1OB1EUmTGJ4xgXL17Ej370I5TLZUxNTeGzn/0syuUy7t69i4sXL2J5eRlBEEBKacorFov45V/+ZTz55JPodrtWW3i9ABAEAcIwxH//7/8dH3zwAWZnZzEyMoJGo4EwDFEqldDtdtHpdNBsNtFsNrG+vo4gCDA2Nobf//3fx/j4OOI4tujDxb+LQ3q+UCggCAKrTf3GgnAIpPRXKBS878RxjG63a+rkdE/9D4LAqpPPLSklgiAw9MefIdy6NORrP6cbfr9YLJr+0/O+Z93+cxzQ/Cec+p53P5VS6HQ6iOMYYdibnfVqjxACpVLJ7isUFPxt4UBj4/IPX93uNd7+UqmU+0yvthMdCCEwNDSEhYUFfOMb3zDlEm0CwPDwMH79138dExMT3vbx8pVSKBaL+Iu/+At873vfQ6FQwBe/+EW88MILGBsbw3/5L/8F169fx4kTJ/Bv/+2/RalUMvyoF+92geZzEASG/vPAbR//Tn313c+DOI6xsLCAjY0NrKys4O7du5ienka1WkWhUPDOwziOsWfPHhw7dsyqK4oiFItFi2fwscvrOwHNA6IDPiac7xHQvObPtVotVCqVjDzm8z6PFgGg1WoZWTzo+HEgvkRz0e2/W6YPv51OB8ViMVf+uuDKe5I7hUIBgQwhkD8n3bbR7263a2SK+yyfky6v4rI+iiKUy+Xcvva73ul0oJQyfJuXn4cLt7x2u214I+FHQQFQcNHCy+V4AGDed2UHv8bbxK+1221ThpQSgQgzzxDEKs7w3G63a8ajWCzm0gCNx/nz5/GHf/iHiOMYMzMz+OpXv4rf/d3/DSPDI3jvg3fxjW98A/v370etVkOpVMKtW7dw+vRpjIyMII5jM3/depRSRlYSn+lF23lAuhHHo1uP+9uHY3cMSB9z77k8wFc+b3/evCOdSyll+I5b5yB4cOuncrm+1U/n8PWB65uuvpM3Tj7c9ptX/fDEy3Hr9fXPB9Qfmn957eZ2AOdFURQZ/sHvtdttvPzyy7h8+bKRJ/SXJxeiKMKpU6fwL/7FvzDj7psjUsqMDuTigNfVy/YQQiAMQzOG9Pd//h+/1xNvBAMbtv2AGsMVWwLOMN17xKiJqLmgzjPUOFKoTFKIfROa3vcRFiGw14DkMR4+YKVSCZ/+9Kdx+fJlvPnmmzh//jzCMMTw8DCGhoZQKBTQbDbR7XZRr9fRaDRw8OBBPP300xgdHTVM32VE/YDjz8V5LwPDFUg+XOX1n9/nY8qFLzdYhBA4ceIEWq0WhoeHMTY2BiklFhYW8Oabb2JnZwelUgkTExOQUmJ5ebkns/Hhp1wu48GDB6jX60YpBbTB1+l0LEZHYw4AY2NjiOMYS0tLmJyctOrgjLaXMOUKUB7kGYzdbtcyPAl3ecKG7pFwJIOAzz3fHOhnHH1U4HX2U2J979IfGV55Qpd/50KWjHyXbzyswtHPoOrVB3I4fNT3uaDo9yz/9JURxzGKxSKKxSLCMLSEGin9d+7cMXOwV/mE23a7bfo5NzcHpbTBe+TIEdy4cQPvv/8+fv7zn+Oll14yz1J7+iltvB56Ps+xwfvKf9M7vZS2XuVxBSIMQ7RaLeNkcXky4ZHX/TD1DUKL/YzQfvW5fGDQ9+jZKIosB+sgtM1xQnKY18MNhkHKIb426PO8LvqulMo1anuVR7yfOw19/MSHH/6dHKlc7vQDl/fTXCAe2c+o5fd4Pyy9AgJK9Ha20LOFQiEzP108uM5Itw9BEBijAwqJYZ3l03l9Iactd+C5xhrJUdIptre3USgU0Ol0cOHCBYyNjeGlz76EN15/Az/5yU/QaDQQxwrPPPM0vvSlL2FiYsKaz3m078pnjod+Y0P3iZ7y5q77O+85Ps+pfJ9+2w98zw5CZ7zOPH1lEODzK6++QdvBDW2uu/l49f8q4H3nNOPSKm9rP6D+BEHQUx5S/a5uRLyLl0WO3xdffBG3b9/G+vp6xsDMg+npaZTLZa+u00ve+H67OgmVwcv7h8L/MsN2EPAxCkK6j1nxZ/h31+p3jRAXuNHslsW9AkC+stKPCUdRhNOnT+OFF17Aj370I3OtXq8jCALMzMxgbm7OEvCdTgdHjhxBs9nEyMjIQynXLjPjDNdHYIMwTN5Hl4HnTV6fQUxOAlrJAvTq62OPPYbt7W0j8IeHh3Ho0CGEYYiRkRHUajWsrKzgwYMHFlOgseZ1kbDk9V6/fh3r6+tWe4rFIkqlEprNpqW0kPI/OTmJTqeD69ev47HHHrM8Upxe8hQTLkzcNvpw5BuDUqlkjFq3nDyh5jIv3h5eNp8TpJy4Zbnv+crqRf+u8e/iIW8+0Xe+yuG2Ie8a4cAVYr3ANf55eXlj6OuDK0QGqT8Pf/T+IEo81ZmnVJFzpVqtGidSp9NBq9Uyz0RRhNu3b+ORRx5BuVzOlOUatVtbW1hZWTFKA60AAdqZRDT4wQcf4Pnnnzdt7DdvfHggA7PXir1LY1yJALJ49PE4Dt1uF+12G81mE1JKjIyMmBUmH/Sbl/2UqEEULNcwcD/z+Ard47jpZfj56gVsWh5UyeDziq/a9jL88ngEGUFcLxiEl/raEYiUlgZRurmxRrIrL5KFO+N4v0iGkGPJdTDxunxGEf/rdDoAtPzM63MeEA59vKUXn+M4JzlLegw9Q99JppDi7NMzLAPD0/csTWR1Co4bahsvNwgC3Lp1C//X/+v/wuLiIhT0/a2tLczNzeFv//ZvcenSJQwNDeGZZ57BK6+8gvX1dbz22ms4ffo0Dhw4YJxyvP+uDCQjzOeoGXSuUDnuamc/485Xlys/fPqYr/68+e++4849nz7otqnXPOulX/MyfHW69bj1uZGRnB/yej6q4dSLn/qAG+w+Xj1IOdQn4xjKaRfRJOcZ7rwiXVoIgcnJSezs7KDRaHjbwNsnhcTk5KRXLvvqcO/za7SAR9Fk7hgNwqMHgf/lhm0/YnYJjyMjz6vrUyTd+vKYJDdmXKZNK7VuPb4+EVHkAXlmP/nJT+L8+fPY3t5OQ2+CAL/0S7+ECxcu4MKFC4apNZtNDA8PY8+ePbl19jI4AOQKLp+g5OW4xORTMvIMGy5gXKOErgEwXv8oiqyVEP7eM888Y97vdDrY2NiwjEtqG1/RJiBjsFwu45133sGPf/xjbGxsoFarmTaSUkFhkqQAK6UNSiklKpUK3n//fWxsbGBiYsJ4f/PwwfGYp5z1+k3vdjodlEolK0zM956Ld94Pd1XJHSsexcCjGvi4+kKo8wweX995tEMvhdkHnLnxPvZi4vy76/njbfaBb7w+qqCjdtMKl+9ev2t588gH1F+XDoUQxnFTLpdRqVQwMjKCubk5TE5OGkFC4Z1ra2u4e/cuTp8+7VVGwzDExsYGzp07h1defgWra6uYnp7G6uoqJiYmMDY2hp2dHdy9e9e8Ozc3h9XVVVQqlYwj0RdNwtvvU2zcfvdSUDheBlE++H1yvrXbbXQ6HdRqNayvr6NcLmd4MH+/2Wwavsav96L/PCWT94nmdh5fcfHkwyXHlU95cA0Et6xeSlQecNyTfHOVy0FDFguFgjEy6LmHUXTo2W63i1CGAPJXr/OukczJ0y98NMGNLfp08cBhELonpZbKdGWtO3/c94FUwabVHzIvB+HxXCfjIc337t3DxYsXsbi4iOnpaXz2s5/F0NBQhj9x/MVxDBXHgBOdY9Ew0lV2V2fhK50cp7T16Y/+6I9wb/GeKateryMMQxw5cgRRFKHRaKDVauHtt9/G5uYmgiBAo9HAD37wAzzzzDNGRxgUBo1C8EEvR8Agv33jxXHda471kzH8vUF1GreMXs6xXvUTnQyKW197eZSBT5fKm7t5smJQXujaNT5dhvPdPDnl3qff5HB1tyNwOvLhjp4l5/T29jampqawvr6Ozc1NhGHYs/5iqYjx8XEv33HrGAR/JCNcXsbvu3rtw8qAgQ3bhym037t5IV4+Jk2KY6+9A/Qcv8+/5xlEeYTlKk6+Z1xQSu9vmp2dxdNPP42XX37ZGEiNRgPtdhtf+cpXTAiAlBLb29vY3NzE8PCw8Q73U35cPOat9LhCgb73YLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Create a bounding box annotator object.\n",
|
||||
"bounding_box = sv.BoundingBoxAnnotator()\n",
|
||||
"\n",
|
||||
"Now comes the fun part. Let's wrap up our code by utilizing `Annotators` and a `VideoSink` to draw bounding boxes and save the resulting video respectively. Take a peak at the final code example below. This might can take up to a minute to run, since we're processing a full video. "
|
||||
"# Annotate our frame with detections.\n",
|
||||
"annotated_frame = bounding_box.annotate(scene=frame.copy(), detections=detections)\n",
|
||||
"\n",
|
||||
"# Display the frame.\n",
|
||||
"sv.plot_image(annotated_frame)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "o8SsyCid6YV3"
|
||||
},
|
||||
"source": [
|
||||
"Notice that we create a `box_annoator` variable by initalizing a [BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#boundingboxannotator). We can change the color and thickness, but for simplicity we keep the defaults. There are a ton of easy to use [annotators](https://supervision.roboflow.com/latest/annotators/) available in the Supervision package other than a bounding box that are fun to play with."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "gKHSYO14vGC7"
|
||||
},
|
||||
"source": [
|
||||
"## Saving Bounding Boxes to a Video\n",
|
||||
"\n",
|
||||
"Let's wrap up our code by utilizing a `VideoSink` to draw bounding boxes and save the resulting video. Take a peak at the final code example below. This can take a couple minutes deppending on your runtime and since since we're processing a full video. Feel free to skip ahead to see the resulting video."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "Es4SUdPGqlF-"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import supervision as sv\n",
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"from inference.models.utils import get_roboflow_model\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"# Create a video_info object for use in the VideoSink.\n",
|
||||
"video_info = sv.VideoInfo.from_video_path(video_path=path_to_video)\n",
|
||||
"\n",
|
||||
"if __name__ == \"__main__\":\n",
|
||||
" # Download the video asset from Supervision assets.\n",
|
||||
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
|
||||
"# Create a VideoSink context manager to save our frames.\n",
|
||||
"with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
|
||||
"\n",
|
||||
" # Load a yolov8 nano model from roboflow.\n",
|
||||
" model = get_roboflow_model(\"yolov8n-640\")\n",
|
||||
" # Iterate through frames yielded from the frame_generator.\n",
|
||||
" for frame in tqdm(frame_generator, total=video_info.total_frames):\n",
|
||||
"\n",
|
||||
" # Initalize the bounding box frame annotator.\n",
|
||||
" box_annotator = sv.BoundingBoxAnnotator()\n",
|
||||
" # Run inference on our frame.\n",
|
||||
" result = model.infer(frame)[0]\n",
|
||||
"\n",
|
||||
" # Create a frame generator object from video path.\n",
|
||||
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
|
||||
" # Parse the result into the detections data model.\n",
|
||||
" detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
" # Create a video info object from video path.\n",
|
||||
" video_info = sv.VideoInfo.from_video_path(PATH_TO_VIDEO)\n",
|
||||
" # Apply bounding box to detections on a copy of the frame.\n",
|
||||
" annotated_frame = bounding_box.annotate(\n",
|
||||
" scene=frame.copy(),\n",
|
||||
" detections=detections\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Use a VideoSink context manager for saving frames of a video.\n",
|
||||
" with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
|
||||
"\n",
|
||||
" # Iterate through frames yielded from the frame_generator.\n",
|
||||
" for frame in frame_generator:\n",
|
||||
"\n",
|
||||
" # Run inference on our frame.\n",
|
||||
" result = model.infer(frame)[0]\n",
|
||||
"\n",
|
||||
" # Parse the result into the detections data model.\n",
|
||||
" detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
" # Apply bounding box to detections on a copy of the frame.\n",
|
||||
" annotated_frame = box_annotator.annotate(\n",
|
||||
" scene=frame.copy(), \n",
|
||||
" detections=detections\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # Write the annotated frame to the video sink.\n",
|
||||
" sink.write_frame(frame=annotated_frame)"
|
||||
" # Write the annotated frame to the video sink.\n",
|
||||
" sink.write_frame(frame=annotated_frame)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "EXTE3zWxqlF-"
|
||||
},
|
||||
"source": [
|
||||
" Notice that we create a `box_annoator` variable by initalizing a [BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#boundingboxannotator). We can change the color and thickness, but for simplicity we keep the defaults. There are a ton of easy to use [annotators](https://supervision.roboflow.com/latest/annotators/) available in the Supervision package other than a bounding box that are fun to play with. Next, we create a `video_info` variable to pass information about the video to our `VideoSink`. The `VideoSink` is a cool little context manager that allows us to `write_frames()` to a video ouput file. Prior to writing the frame, we annotate a copy of it utilizing `box_annotator.annotate()`. Let's take a look at the resulting video. It will be installed locally and is called `output.mp4`. When run, you will see bounding boxes around the detections. Pretty awesome! "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We only scratched the surface of all of the customizable Annotators and additional features that Supervision and Inference have to offer. Stay tuned for more cookbooks on how to take advantge of them in your computer vision applications. Happy building! "
|
||||
"<div style=\"display: flex; justify-content: center;\">\n",
|
||||
" <video controls width=\"1000\">\n",
|
||||
" <source\n",
|
||||
" src=\"https://storage.googleapis.com/com-roboflow-marketing/supervision/cookbooks/annotate-video-with-detections.mp4\"\n",
|
||||
" type=\"video/mp4\"\n",
|
||||
" >\n",
|
||||
" </video>\n",
|
||||
"</div>\n",
|
||||
"<br>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In the code above we've created a`video_info` variable to pass information about the video to our `VideoSink`. The `VideoSink` is a cool little context manager that allows us to `write_frames()` to a video ouput file. We're also optionally using `tqdm` to display a progress bar with a % complete. We only scratched the surface of all of the customizable Annotators and additional features that Supervision and Inference have to offer. Stay tuned for more cookbooks on how to take advantge of them in your computer vision applications. Happy building! \ud83d\ude80"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "V100",
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
@@ -197,9 +392,9 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.7"
|
||||
"version": "3.11.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
+25
-3
@@ -17,7 +17,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
@@ -45,6 +45,7 @@
|
||||
"source": [
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"\n",
|
||||
"# Download the a video of the subway.\n",
|
||||
"path_to_video = download_assets(VideoAssets.SUBWAY)"
|
||||
]
|
||||
},
|
||||
@@ -52,7 +53,28 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We're now equipt with a video asset from Supervision to run some experiments on! For more information on available video assets, visit the [Supervision API Reference](https://supervision.roboflow.com/latest/assets/#videoassets). Happy building!"
|
||||
"With this, we now have a high quality video asset for use in demos. Let's take a look at what we downloaded. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<div style=\"display: flex; justify-content: center;\">\n",
|
||||
" <video controls width=\"320\" height=\"240\">\n",
|
||||
" <source\n",
|
||||
" src=\"https://media.roboflow.com/supervision/video-examples/subway.mp4\"\n",
|
||||
" type=\"video/mp4\"\n",
|
||||
" >\n",
|
||||
" </video>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We're now equipt with a video asset from Supervision to run some experiments on! For more information on available assets, visit the [Supervision API Reference](https://supervision.roboflow.com/latest/assets). Happy building!"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -72,7 +94,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.7"
|
||||
"version": "3.11.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -0,0 +1,566 @@
|
||||
{
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"provenance": [],
|
||||
"gpuType": "T4"
|
||||
},
|
||||
"kernelspec": {
|
||||
"name": "python3",
|
||||
"display_name": "Python 3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python"
|
||||
},
|
||||
"accelerator": "GPU"
|
||||
},
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# Evaluating Alignment of Text-to-image Diffusion Models\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"[](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/evaluating-alignment-of-text-to-image-diffusion-models.ipynb)\n",
|
||||
"\n",
|
||||
"Click the `Open in Colab` button to run the cookbook on Google Colab."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "pviK1VFFNp8h"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Introduction\n",
|
||||
"\n",
|
||||
"It is a common scenario to evaluate text-to-image models for its alignment to the prompt. One way to test it is to use a set of prompts, consisting of number of objects and their basic physical properties (e.g. color), to generate images and manually evaluate the results. This process can be greatly improved using object detection models."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "FtZXoFjPt569"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Before you start\n",
|
||||
"\n",
|
||||
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "1yOkAwH-UYPM"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "MVfLwWd8D1ne",
|
||||
"outputId": "70562805-224c-41ad-cf46-f6808f469923"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Thu Feb 29 18:16:26 2024 \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
|
||||
"|-----------------------------------------+----------------------+----------------------+\n",
|
||||
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
|
||||
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
|
||||
"| | | MIG M. |\n",
|
||||
"|=========================================+======================+======================|\n",
|
||||
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
|
||||
"| N/A 46C P8 9W / 70W | 0MiB / 15360MiB | 0% Default |\n",
|
||||
"| | | N/A |\n",
|
||||
"+-----------------------------------------+----------------------+----------------------+\n",
|
||||
" \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| Processes: |\n",
|
||||
"| GPU GI CI PID Type Process name GPU Memory |\n",
|
||||
"| ID ID Usage |\n",
|
||||
"|=======================================================================================|\n",
|
||||
"| No running processes found |\n",
|
||||
"+---------------------------------------------------------------------------------------+\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!nvidia-smi"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Install required packages\n",
|
||||
"\n",
|
||||
"In this cookbook, we'll leverage the following Python packages:\n",
|
||||
"* [diffusers](https://huggingface.co/docs/diffusers/en/index) for image generation pipelines,\n",
|
||||
"* [inference](https://inference.roboflow.com/) for running object detection,\n",
|
||||
"* [supervision](https://supervision.roboflow.com/) for visualizing detections."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "xg5JFGa4UjZq"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!pip install -q torch diffusers accelerate inference-gpu[yolo-world] dill git+https://github.com/openai/CLIP.git supervision==0.19.0rc5"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ri8GPZ3SD7LT"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Imports"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "U0qBpU5YU3P9"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import itertools\n",
|
||||
"import cv2\n",
|
||||
"from diffusers import StableDiffusionXLPipeline\n",
|
||||
"import numpy as np\n",
|
||||
"from PIL import Image\n",
|
||||
"import supervision as sv\n",
|
||||
"import torch\n",
|
||||
"from inference.models import YOLOWorld"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "sBKUQMdwJTv1"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Generating an image\n",
|
||||
"\n",
|
||||
"We'll use [SDXL](https://huggingface.co/docs/diffusers/using-diffusers/sdxl) model to generate our image. Let's initialize our pipeline first:"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ZzEhndAyU_EF"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"pipeline = StableDiffusionXLPipeline.from_pretrained(\n",
|
||||
" \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
|
||||
" torch_dtype=torch.float16,\n",
|
||||
" variant=\"fp16\",\n",
|
||||
" use_safetensors=True,\n",
|
||||
").to(\"cuda\")"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "qCwxwGcuEHJ-"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"In this example, we'll focus on generating an image of a black cat playing with a blue ball next to a parked white car. We don't care about the aesthetic aspect of the image."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "XrQ8ose0Vm-T"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"PROMPT = \"a black cat playing with a blue ball next to a parked white car, wide angle, photorealistic\"\n",
|
||||
"NEGATIVE_PROMPT = \"low quality, blurred, text, illustration\"\n",
|
||||
"WIDTH, HEIGHT = 1024, 768\n",
|
||||
"SEED = 9213799\n",
|
||||
"\n",
|
||||
"image = pipeline(\n",
|
||||
" prompt=PROMPT,\n",
|
||||
" negative_prompt=NEGATIVE_PROMPT,\n",
|
||||
" generator=torch.manual_seed(SEED),\n",
|
||||
" width=WIDTH,\n",
|
||||
" height=HEIGHT,\n",
|
||||
").images[0]\n",
|
||||
"image"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 817,
|
||||
"referenced_widgets": [
|
||||
"80345cfe8a724651a287a20078218742",
|
||||
"b79d4e509fa54f899391ff0ca80730c3",
|
||||
"16cbb7e6ec6d444e8d71042afaf98806",
|
||||
"4e3ebed950c945a0a7d74acaf8a4e071",
|
||||
"0c2fc95f4da4409d8789c1086efc7487",
|
||||
"a677a168d3dc46f48a6bd9877bf2873a",
|
||||
"df3cb55850ba40afb692bca7f7557bbf",
|
||||
"abb0cd4b86ed4d1bbb5bede80355be74",
|
||||
"b403cba7f1ff4cd0a4a850de46bc7d3e",
|
||||
"40ee234278f0466587191689f3a4e77d",
|
||||
"48df83b840204738a2530fc222a17dc9"
|
||||
]
|
||||
},
|
||||
"id": "TaWaIpjQERtf",
|
||||
"outputId": "236e6107-b50f-4eb8-8db8-234519245264"
|
||||
},
|
||||
"execution_count": 41,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/50 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "80345cfe8a724651a287a20078218742"
|
||||
}
|
||||
},
|
||||
"metadata": {}
|
||||
},
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<PIL.Image.Image image mode=RGB size=1024x768>"
|
||||
],
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaAAEAAElEQVR4nIz97ZbkyI4kCAo+lDT3yLzV1T3vPO+xv/bJpuvsnpmum5nhbqQCkP0BJc0ib/U561U3w8PDjR+qSqWIABDI//n/+n8bK0L+TINvNjbOp3z9c5ena5iUaVYVVczMzCHCLMlQFewfZTvHR8og9lnbmfsUx/ZhqlJUYrPcDf/j9/2H01z/OvGfU38+q8qiNKlFQISEgEIBACEAkiBUBYQISIgISRERQFSqSgACIoL+BlARkqpSZP8+BEJABICs31p/kFBdxyTRX30uXecCCChY7AMI1s9JiIKE4NcvQlRI3idBX/n6xz490fcFkOzf78vs3xK5r/TtYwTWOLzOtsYEb9dzjRLe7qiu44tgnes+vgCvT4EFUbBPep+2J+g6L18z0ud9G6s1zuCv1/86fv9+rYG75o6gXONwTwOuY75fSU8j1/fXyIgCxR6fexJf47MuCmtkrisUFfT4FyCv8RBd90ugZ1xV+ziyrr9EIBBTAakKEKZ9bKiKAMXqzwLoRV5FUe0r65m65vRaXQIWe81DeF0RVAVF0XXzAmivSVBUBQBr3a2ARUqPhmQVi2paxagKxqyzIpDTXQw2oCg9U448M/9C/eF8Mr5+fv1T4xgZkVrjQ8dmSMb3PP55fv2v8/yOYiTreM7nsyKzahvugorMyoj6+fNI237/t/9mAmR+/fnP4zh8c3VXFSkO16o8jnMGv8+apb9/Pn5/bBKnoGTo1znPMxQ5XImy4YDETBECGRHzCBEb2xDgODMTYxhZ5zGhIioZOSch2lNWmSJSlWrKQmWZq5kJxFwFklkEWTVcqgoQcy+yCgL4UBbnGVWlCh/GLB+eVaYmppUVmSpw13lGZomZmw4fAlGTiIg5zf08Yt+HDwOKZEZs+y6qERlR/YyN4cMVpI8xhqtKJb+/Dxv6/dchpmOMMWzOeD5PVVFVFenlFHOqiftQwXkcP38eahZJioqImwhobiTjDBv6+NhNZc4kxM1JEohIVXnsw0wyEyx3J3GeUVnu6m5qSuLPf34BfHzsH5+fUMzjMJN5HNtmHz/24T7P+XwepmP7+DC35/cZNZkVM8z0x49PQo5jzllZgOjYx/PrO+dhDhAfj8238fn5cJF5HvOc2+ePfd/nc37+2M/j/Pn1rSqbj23fnzP/+PPrz7++zjl//+1zG87ijEpC1FnMrDjPeU5zFZHM7Kdsc1NXEFWMYBazSkTcXBWZNWeIwt3dDSLnmWM4q8y01haBzBQws8bmPSmRkWeqqZjuHw8QzIg51VUhM8rc+9GOGcfzaULffBtuZlUVUQVRFVNRETUV0aqKCFMjpDJMZXvsxayIijQVE5qZqANynlNUWaVCHzYee0/u9/choEIEULOCZFZGQjDcWJEZBKqkCttjF1Cqqso3z6gsokiUuldkZYnKtjuqFLJ97AQFECCjsHZOg1oRc86McLfK2jYrcp6RmWJqKm7mrqojCTVVkcoQUEWqmJXzDDH1bYAUiKiokhEA1C2iqrgPH8NFDarMGpsLhChWZkZlqDkpSani58emInNGRpo7IBSSVAUzK2O4Pj4e+/ZRIoCYgBm+bRDM7+e2mQ8bu8ecrBoGFGKGbWPbR2bEOUFsY/dt2NgiA6iIaSKiKqS7bfu2+aAwK8/nt1R9fn7s+6f6OJ9nMcY28jzVhAyF2/a5f3ya2fdff53HT1URIUQp5vteRRbzfA73KorKGAaBqoGq/sP3ByGM4/z6z+fzCcmqBFhFc62EuZlID+7j40PVquJ8Tt+H2jAfY8jx9T3PQ03NrZisMnG1jTQ1A3K4i6raNvYfInJ+PwUFVBEi2pMCiO8fmYiSKsb5FFblCREoTKXfpGP7UB3mHyKGYkYRpSpkkFVVZgPQ8djmmaogCwWIQrWfEZIsiIgYVExg1CEKRqhaJYqaSVUjiWJE/vz5/TwOkI/HY9sfPkbDTyyQQBVlo0ws6LLAwwuiEUQBrAJvhHZBmgtXiWrDlv7LBeEaBTeEoDTSAwGqClmQEoQos6ZKKEJdRLVYcU6B2diSyCKzWCBBF5BMGiSlPDMdglnbNrg/4I9CyuaZQBUZgROosQ9zs00KqJlAqrsaymUyVYxSAkLuK1NRRZSkkDKfwEOH2gCUJSSrAF1gF1wATQgutC0CAVgl0Av5NcosAqjrYxcOXjASgKB44+01Ezfyu5DXwnovnoAbIwIv9AaQUoIb9/c1yAVfX5dwXwqamVxYVShrmuUCdIS8OMlCt3LjwRs14nVRr28ad143upAwANH7BiAvpnNh8b5irt+vi0fdV/86jqyL5XUUuRA/rjGV60qukcfbwbgI3H0FJNBL4h3Iv87bT5Ss4X5d0MVh1oX0La9HDRTgOuT9g3secR+JVTe97Iu65rH/lRCg3sZ3zeBrZa3HL1NE15GLr/VwsSkBqmiqAFnkPbR8TeP1+3hbb+88CbV4Zt/POoXcbGbxBABkQd9ozeI5/TpcWxJJmlgi+xfENKcUlCqcxRSKylBR18avX1+m+vw6lJGV8/mdNc0eqqLDas7j+c15zmNCODbnmUfOWVGRNiyZGVWZTFQVlclZNc3sPM4kt48hqlQBZc5gpZokYa4/3G177PtjqPJJcNrwgSIKxWCA5GRGicj+GGYGMCdlsVsBoQpIZWQDE8mGKyVKpuQMkjZ0cXNQFSKMOQFVczNTNQJVCdYi52DTucyqZFVBoQIRYZWoZGZElpXDIKIqKoioIiFkJUxyBgCjVVURCqoBAhGKSPOQYiGaVbKKVaUqQAllf+i2baoyz7ltPiPJYgEogYnctNPcVV2P86zJbYx9H5nFqfvHdpwJxTCNrMgUkEIf7puLgqwsAMxIdzXVqH7NqCpEpKoywoeLCgRqQjIrj3nOIyE016qcxyHDkgVCXfaPse27qwH551/niZDhouquOSXBQikkMgWakb13kTXPMyIqwnxAuD3Gjx+/DR/ktPF4Ps+IQMlwL9gx+f0dY1cQ6iYm6mKuvz8+x7CcMc8ooErMAUhm9Eu0H6tKQvB4jH3fVCQykSUVeQYAGy5SmYRAtDduZmY/91VZLFRlcIwhqqqaMauqWExko1KTqgLr+H6aKVDFYkA3H7uZaWMCH9u2GzN7r8jMIokS0cxSsaoCkZyZZW5QkEVhkjNDVKBC4Rn5+bE9Pj5V7OfX95zTh1cVlIRUZUMEkUtKSNQlvajJGC7C53fNmWPYGLZtQ21kVsahgsrKzIhSU3eHSOtZUBBi6gLGjCqayba5bkNU8gxIq1HFKgBVZS4AkIWLLUQki2qj5rl97L6NeSZZGdnkhxAxQatsJMFhjuKFpVns5azFMjEB1cRc53GqKURYpaaiso0RlDiTZLLmGRFhxbENc61Mc0lIJWGqZmLYxjbnnHOa0obY8IpTTYqckTGDoLuLcntsjx8fCZ5fMTNVNaQq8uMhrvb8ms/vw10en4/znD4kI2t4VUVWZqmWuo3NoOL78G0/ns+v53OM8dhdVWeEV+2Px9j9PPD9PMcw3yUrJNTH5h/jlDi+TvMx3GAK4vvrcPfN52P/XU3PZ1SYnnrGFFOyYEiSKu6qpjnposWWmMRcIGLu/bYqUNRElYCZz5xRfOzGEh2m0PM4zbdNAUpRi9KrvWbs+1DXSkZUfD8B9W0f264o1pxQCCCYkSz68CqoqpmrjTzmcBOVzIiqKvrYAKmSeSwBpV+kaipqgcqs9b5UYTEyzcVcCVT1O9QYFNEqgmCicu0TEL0Bg9jSfVkLLYlIY9KFROpdUL7e4wD50jf5Djt5wQ65RL8Lwq23zMJBuCjBgnGqEG0u9RLBIVJgFSHa9wUxCksqIwj1bWOyKnSAKY55zoKYu2YhRKZuEDpOq29WTJNQh6GMKSdVVAkfBpbWtHKAWQmpEoQqxJLBREFNQEFW/XGck/IbXVX3TU9aHmQGZPC+MTQduMcIApj6el3frEreEFEDwQZqvPDmUqZfavuFTtc5LkR2//SeqkXOljp/w94L7t9xgxsbLtDcIPdXsfmlKBNUuU7baPcViFhxgLej89aFF7AWyL2krtu6OEx/9l4fd+SAeA3QC4L2/93MYI3m+sc30Po2Wq3DvziMyHWP/ZNGwK91fJ2Ka0rkmi9Zs/Uav+tCb9h+xXkWw+E1v41rLywOuf9HvF/nC1h3yIW6Ai8vdvMiHtcSaHrSswARroUmTQ7k5klrumQ90tcpGuUJ4ap94XIfdq1iFimitXaNehGV3kJ6zwBEtMkW+es81FuwAItGCgDUCpLICgvcmwBeS7tUpVhFzmQEk4hjSqYAvo9S1RIIlVRKHpPfz8yfn6PmsPPrp6hKHvWsipk5cwarxHWMTSSeXyZiYohIFZ3PmZkmum2+79tmNobsYwzV7SHHcRTEzBpyxvnUgoqomo1t7I9t3w0VZefzzDlNxM0IQXHb3Fy/f36riYqOsZkPs/P76yhWZWVvhKH9furQh4iYaRWI6sc1swQKXMpiUaQIRiQIM73WjxLMCklVs95z54x+pNzcXfvJIzvAUpViw81URcgpIqpW1dMqkVlRVcxMEYwxtm34ZgJGyJwzzrg2cQHQEadMmtl5ho1UamQWeZ5nkQrOGZEpIr5ZzmQF6TErIwHMGWIGytge5jnjK85MZGWJUFRJVNJUI2OxfEhrsWaaWaoqfRyRrFLTpbwOFyAj5zyvmAkrKEOSNb8mkdvmH499ezx+/P6bkPOPJ1TOc+pxzEhzPc+ZkQ2mzzmHe1ZUrHliQkRsDIjuu7v74/NhqnHmX399USiC/WP7/Pgc2+O33//x47c///rrn5nz6/ktOkB8fu4qKiIzGZWkVKGK5IpsqCrRATcVFXc3s23fe+ng6xmRVSTrOEtF1K1hdLP0Zg/nebIRn7uoRmZGRISIxMyxGaBmCoFkVVVlghSFqJoKKyBSpLine truncated
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 41
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Not bad! The results seem to be well-aligned with the prompt.\n",
|
||||
"\n",
|
||||
"## Detecting objects\n",
|
||||
"\n",
|
||||
"Now, let's see how can we detect the objects automatically. For this, we'll use [YOLO-World](https://inference.roboflow.com/) model from `inference` library."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "wKuyU-GDXPhr"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"model = YOLOWorld(model_id=\"yolo_world/l\")"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "R9pERjDkGIlN"
|
||||
},
|
||||
"execution_count": 42,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [],
|
||||
"metadata": {
|
||||
"id": "G0PlnmZckOCH"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"YOLO-World model allows us to [define our own set of labels](https://docs.ultralytics.com/models/yolo-world/#set-prompts). Let's create it by combining lists of pre-defined colors and objects."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "VOxtES_qaoWe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"COLORS = [\"green\", \"yellow\", \"black\", \"blue\", \"red\", \"white\", \"orange\"]\n",
|
||||
"OBJECTS = [\"car\", \"cat\", \"ball\", \"dog\", \"tree\", \"house\", \"person\"]\n",
|
||||
"CLASSES = [f\"{color} {obj}\" for color, obj in itertools.product(COLORS, OBJECTS)]\n",
|
||||
"print(\"Number of labels:\", len(CLASSES))"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "SkY3TI4JHmkj",
|
||||
"outputId": "253a95a8-0c32-403a-db63-82e529de2eba"
|
||||
},
|
||||
"execution_count": 43,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Number of labels: 49\n"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Let's feed these labels into our model:"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "by_NYnqkoET-"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"model.set_classes(CLASSES)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "oinXZU3YWZqw"
|
||||
},
|
||||
"execution_count": 44,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Time to detect some objects!"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "RTHHpRsFoJsk"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"results = model.infer(image)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "zDPd-LOqGLHJ"
|
||||
},
|
||||
"execution_count": 45,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We'll convert the results to the [sv.Detections](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections) format to enable features like filtering or annotations."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "TGCRxUpeu8Vi"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"detections = sv.Detections.from_inference(results)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "GWbmFKscvUhU"
|
||||
},
|
||||
"execution_count": 46,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Speaking of which: we only care about strong detections, so we filter out ones that are below 0.6 confidence."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "AscEY8SioQQ7"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"valid_detections = detections[detections.confidence >= 0.6]"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "f2fvqc8lWxGT"
|
||||
},
|
||||
"execution_count": 47,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"A quick peek on the detected labels and their score:"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "klstslXPop-9"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"labels = [\n",
|
||||
" f\"{CLASSES[class_id]} {confidence:0.2f}\"\n",
|
||||
" for class_id, confidence\n",
|
||||
" in zip(valid_detections.class_id, valid_detections.confidence)\n",
|
||||
"]\n",
|
||||
"labels"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "0jbyOEGdGOhd",
|
||||
"outputId": "a7a90eb0-d829-40d3-8090-151e2c735d8c"
|
||||
},
|
||||
"execution_count": 48,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"['blue ball 0.95', 'black cat 0.72', 'white car 0.68']"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 48
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Visualizing results\n",
|
||||
"\n",
|
||||
"Now, let's use the power of `supervision` to visualize them. Our output image is in [Pillow](https://pillow.readthedocs.io/en/stable/index.html) format, but annotators can accept the image to be a BGR `np.ndarray` or pillow's `PIL.Image.Image`.\n",
|
||||
"\n",
|
||||
"Time to define how we want our detections to be visualized. A combination of [sv.BoundingBoxAnnotator](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [sv.LabelAnnotator](https://supervision.roboflow.com/latest/annotators/#supervision.annotators.core.LabelAnnotator) should be perfect."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "GOccbqMDo2Ec"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"bounding_box_annotator = sv.BoundingBoxAnnotator(thickness=2)\n",
|
||||
"label_annotator = sv.LabelAnnotator(text_thickness=1, text_scale=0.5,text_color=sv.Color.BLACK)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "X9HMMnTppp-c"
|
||||
},
|
||||
"execution_count": 49,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Finally, annotating our image is as simple as calling `annotate` methods from our annotators:"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "u3y59TgWp4Fx"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"annotated_image = bounding_box_annotator.annotate(image, valid_detections)\n",
|
||||
"annotated_image = label_annotator.annotate(annotated_image, valid_detections, labels)\n",
|
||||
"\n",
|
||||
"sv.plot_image(annotated_image, (12, 12))"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 734
|
||||
},
|
||||
"id": "UdDEwjhoIiHo",
|
||||
"outputId": "c56bd631-653c-4fd6-cc41-653abc49b0e7"
|
||||
},
|
||||
"execution_count": 50,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
],
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAALNCAYAAAAbaP0uAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz925bkSK4tik3AzOgeWd1rrX2+Ro/6FP2MvkNP+jU9SPvs3d1VmeGkmQF6mICR7hFZlWufo6Gxxgj2qM4IDzppNNoFE5iYEHd3fB1fx9fxdXwdX8fX8XV8HV/H1/F1fB1fx3/RQ///3YCv4+v4Or6Or+Pr+Dq+jq/j6/g6vo6v4+v4P3J8Aduv4+v4Or6Or+Pr+Dq+jq/j6/g6vo6v47/08QVsv46v4+v4Or6Or+Pr+Dq+jq/j6/g6vo7/0scXsP06vo6v4+v4Or6Or+Pr+Dq+jq/j6/g6/ksfX8D26/g6vo6v4+v4Or6Or+Pr+Dq+jq/j6/gvfXwB26/j6/g6vo6v4+v4Or6Or+Pr+Dq+jq/jv/TxBWy/jq/j6/g6vo6v4+v4Or6Or+Pr+Dq+jv/Sxxew/Tq+jq/j6/g6vo6v4+v4Or6Or+Pr+Dr+Sx9fwPbr+Dq+jq/j6/g6vo6v4+v4Or6Or+Pr+C991F898f/+//h/ormgD0G/vcG3v8F9Ao9/oPR/QucDxToEByCO7b6hbRukFsw5gTFQVVHaHVbf0NHgusFwx8PuOHCH6Q1SGxQKmY6bGDYV/Ptbxb/fFaUo/jEc/3g4Hh0YXjBdYRAAAjj4HwB4/CCAxO8CPX+XyzkAIALA4QBEBALhz0+94Dzt5VOJvz19Lq8/OCTbuc7Ptr1cUgTys8u9HO6X64hc/8AP87OzO+Aiz8/+cn2HXz6XpxPysp83SP60rR+vfl6Tzf/zb35+8L1dn/35WS7XfnpmWS356V3/oj1x50/b/affXM+b34+Pn8Zj/OPnV85x+/xOPtxTZJ0rL18WubxICERymOTTACqAPv0NKBqjVy5tdLbZ454OwDynlsc1X9/GOf5VruPFY96dzyKCsw0IL5yc56/z4ieHw9z4swMW/TkAPEZHtw6fB/x4YCuKWhq20jC9Yp+Ox+OfEPufGD/+B8Te8dj/BX//A8UmSvsN8vY3QID+/Z+Y+79wfP+fcOmQWrE/Or7//k+MxwG4Q4tApsPmgE/2x/txwErD3//2bxB37D/eYdbZ4UWhLhh7R1GHKHD0CRGF1A3b7Q1brcD+gM8DuhU8+oF+dLhNuE+oCmxyEdxuDbUq9v3A8Rh8r6qYBsxhkCIwM4w+OJwcmNMwhgOicHPYnJDCQap6jhtRgZtDtWDbKlTpHzU3iBvcDOaOWitcBDbP92VugBtEBCocd+6OOQ2lKrSUc7SYY84Jj3faaoVAOV9EMOdgW9zRakNr/JvNGXM72mWOOR1mhloLShGoKH777Rtu9w1wx7F3HEfH47HDHLjdGlqt6GPiODpUFbVWlKoYNnE8drRWcLvfYNPweOwYfWI/xhqV5gaBo1RFbRUwQBSole2aw6CloNYCM2CMiVoUt1uFiKAfB0SAtm0Y02Bj8soKjDnR98m5WRW1VNTWYHCIGBSOb982fPvtG4ooHo8/8I9//gEtG97efoOqoveO3jvG6CiquN/vgAv2vWMa4C5wEfTeMfuOthWoCv7933/Db7/9DbUUmB14/HhgTEdrN2y1Ybvf8ePHO/74/Q+UJthqQd02PPrA77+/Aw7UVuBj4tg7DIA73/00x+gTc/S1Lo1hgABvbxu2rcVnE2MMPB4dNh2lsm0WC9CcBlGgaIl3gRgrhqIck621NY7mGBBVlCJPK1Zer9QKVYH7hBvbLyoopcQYxjn2zNYaZe6xRrJtrRZInDw5MVBqBURgNqNvKkQFmOyHooK/ffuG23bDYz/w/Y8fa9yX4mitorQNDsd+dNg0qACYAFThwvkKcOzNObE/dogAtVZsW0OpDXMaZj/gbjEecx4XiCpsGuac0CLY2oYiAsBWX21bgSrbPo/BGa+FY/UYXPNVUIoC09DnhE2DFOXcLAVtqyilYLvf2IZjx+wDWhQiimmGaZPvKtadWgvXhDn4jgAcg2tR2xpabdBSIGDfjt6hRbnu9IPvsVaIFozha885HpwftTXUraIU7jECx+gdpSi+vd1xu91Q6obeO2wO1Kq4vd1hYfeKCqQIZh9wON5uBQqOv3rbcPSOx493uBlqayjacP/tDTDD+4939GPH7dagqmhN0WrDdtvg4Dow9nfUKvi3v/8bbrc3mAtEBX0c+P7P31FLwf3eIKpwueHbb3+DCvD993/g/ccPqArqrcAhqPWGtt0gKti//4HZHXWrqFuBQHA8OkopuH37G/7+b/8bRByPH7/jxx//xPv7O0wMaYh7GDfb1rjXT0FrG0ptgDh8dkipqNsdtEUMfX/AneNEVbj+G3C7v8GhKLVB3DDHRG133O7foOWG3jvEByCGMSZu9xv3u8m1X6SgtA2l3TCPAzYP9P5Y9uGcEyKC1m6o9Y7t9htK3WDHpA0swJwdY3ZAuJe6C0S4X4lwXMABjX1vjnHa52nsGed72d7g7rAxIFDYFJhzHXHn3jf6wI/3B47jgJaC+/2G1m4cqznpXuy6C+x5+vtpCec27lzbApfYFTcF/oEAEnOMO9ozfsnve34Nzr3aOYdUHaIG9wGow71D0KGa9ofDpsFNos8KxjTMMWDDoLVCVDH7RKmATcf/5f/6f8OvHL8MbDEGugFSK6pOOA5occhNABTYEPiYUDWUKihu0DkAM1QBahUIDOIdxSoA5wIljioCk4KJAjOBgRPfAHQz/GM/0FHw7cYH3W6K7o7Rw1JwBBBdMGYZ4AFVoVKWUbwAxAsgeQVcnx25gX32vc+g8EfUmrdOoCUnejmtdl7zcq+Exa+tfAJVV8z8Amg/Au3XBvHcD33werrkIH4BV0+Y8RNo6a+nyfVxn9rwETl/1rCX73zS1E+b87PjySmAeCefv7v89M/GS57zOi55K//03M/bdZ7k2V+vXo8P9zz/vobC8kic130er77O5frrEAfKFWzGunZeyuN/sr6f9wQC3F5a53CI5ELpaxX2uEcCmPNVZH9dzo9NW+JGwv87/Th+tkTgmO4Y0zGH02jeO8QMqo6iG2bMQ4GhODD2AXvswHjH26YY2w398QPwARwPAIDZoLMOgGohoDCDaoGUAhuD9+sTow8oBNtWsd02SK1oTVG1oCqwH4JhToAjCrhjHA/AHCqKUiva7Ybb/YZaBBMF+w/H7B0KoJYCqECkorWCfnRMGyiq3ARLg+qOx/tBIDsm5nSol2XQTbMwJhSlOMwQ7yrfofK9qBAUmHOjgmPMieJ5bgBV0Eifk30Ct7gHX1opBa0VnAZQbI7mgNJoEBEYJsyE14AQ9AYIJhgmmN82Ah4aX47esYzM0/XhC4DP6UBxHL1DSvT5nDh6p4GjiuPo6GOwT6rCzeDOvhtjwMzQu0O0QESx3e4Q7Tg6n3vNBI0ZaXQOjDEWsDcHfEy4eTiC2L4xRjiJuBOOMaBaULYGm4YxCF4gAZwmoGoEoDZRquLbtxu2+x3ffvsGN8OPB9ePPjrweKDUghngMGdvHxw3c47Yh2WtCaU2iBDwl1Kw3RpUgP3h6JPG2bZV/O1vf8Pt/g3//h//Dd+//4F//vN/4jgemPsBc8FtqxBRqCqGA4aD480tHBAxFkWX8aUqAToLaiWAb+449gNzGjoG3A3HYVBVaC0otdA5Y77WtNkHjala0GqDCzDHYH+aQx0EQEInOgSQaXA3+Jww1wCzANzgk86zUirbXAgAPcCsiqAqnxXK9ygCiDmmGWwazA0OBLismGNizomCwvZIxXZr+O3v/4bfvv2G7f07zAkUa1G4cF6lgyqGG+aYsMnxqYXzl0CQxnMpBOq1arx/C2NUYQbQCwP4DAPWJox/wNZu+PbtjqoFvR84joN9JXTgCARTLdbBiWnsC/aNwSfXeY13KkpHxXWvy9+nTYw5UFEBBQ1h5xippaDVirpVqAh8ErxCBVomnRWxrwgU5o798cA0RzXO69Jq7Dl8T9um7DufMDFMNxQ45hiAcP1REbRtA3tO0ftAH8Z5CsBsQqJvIYbeDXacjtzjmHh7u6Ntd9Ra6QwwYN8f6EeHFQd+cA/bA+jRWSHsRwEeP96hpdDxUApghuMY2DY6K0opgAO3+w0wh7ugakW93dFawf540Kk1B3waDAXtdkNrdHSMftDRrA63iXEYPDygBB4dIpNgP96HKCDh0M5zixbAHBYGzBgTFue0Vjg3jA6TOQedQsL1iM41GsFjdGhp8DmBsDhqVTpCakOrir5P9NGxtQKJ/WZ2wzCjAwgb1AWlNhw21jwF6LhPQ0JB0D2dY7qosj0zxqfRgV60QpUOtRlrdilKm89z9YxdLmwhKYJSBLUVqBQMEfRjhBM+1461M/I78mpSnwG4ZYmdxt8yl8/vXOxNSbvp+fPnuI9jgeWriXgxeh2ru+L7Z196ttE9bC3QMFPEWDRY94XbJAJ50yZmj3krgNuEhdPeByA4nd5/dfwysLWywQRQcTR0YHCiwSdMBNAC14JyK5AicBhgA/RYC7ZCqMqXXNGEE4YTU+FaIFIxUDABTAgOp1e/T9qVswpaRA5KcdQJwAwK5XVFzhfuyjasTjdcI6GvUTaHQ5epn6DrNcqax/mG01t7Qc2fnHv+u2CAc7Q+Y47LKLrg0udBeEFxfjkx/nQa9TkbnkGh5zOt538e9Ofznp8v/H3pm/OGL33zAW/Jh3/88ijnI8jzh09fk0t/fAInP0Gu13m+PnkFTK+NfXJ4/OS5/OMjPnnlnvru/Pn6/3/S7PMPl0VCRD4sbFdnyDMQfLnHpSMFkqsOMsIqOBfdj69SFnBdkdmzVZc+icU4Xd1P4zKfw+FCyCxPb4WANefjZak+nUjX8RC3ULine truncated
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Testing it automatically\n",
|
||||
"\n",
|
||||
"We can also test if all requested objects are in the generated image by comparing a set of ground-truth labels with predicted ones:"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "kl5kO59CquMq"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"GROUND_TRUTH = {\"black cat\", \"blue ball\", \"white car\"}\n",
|
||||
"prediction = {CLASSES[class_id] for class_id in valid_detections.class_id}\n",
|
||||
"\n",
|
||||
"prediction.issubset(GROUND_TRUTH)"
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "laJS1E76qynK",
|
||||
"outputId": "de193606-b41a-445d-8107-de02218190bd"
|
||||
},
|
||||
"execution_count": 51,
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "execute_result",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"True"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"execution_count": 51
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"Using [sv.Detections](https://supervision.roboflow.com/latest/detection/core/#supervision.detection.core.Detections) makes it super easy to do."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "8L4lXpvFvxZR"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"In this tutorial you learned how to detect and visualize objects for a simple image generation evaluation study.\n",
|
||||
"\n",
|
||||
"Having a pipeline capable of evaluating a single image, the natural next step should be to run it on a set of pre-defined scenarios and calculate metrics."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "j2GgYtNGqYmY"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Vendored
+222
-55
@@ -2,7 +2,9 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "BsraWznGULVc"
|
||||
},
|
||||
"source": [
|
||||
"# Object Tracking\n",
|
||||
"\n",
|
||||
@@ -14,8 +16,71 @@
|
||||
"\n",
|
||||
"## What is a Tracker?\n",
|
||||
"\n",
|
||||
"Trackers are a piece of code that identifies objects across frames and assigns them a unique `tracker_id`. There are a few popular trackers at the time of writing this including ByteTrack and Bot-SORT. Supervision makes using trackers a breeze and comes with ByteTrack built-in. But first, let's get our deppendencies installed. \n",
|
||||
"Trackers are a piece of code that identifies objects across frames and assigns them a unique `tracker_id`. There are a few popular trackers at the time of writing this including ByteTrack and Bot-SORT. Supervision makes using trackers a breeze and comes with ByteTrack built-in."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "7h__vutkUZSY"
|
||||
},
|
||||
"source": [
|
||||
"## Before you start\n",
|
||||
"\n",
|
||||
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "pzm2gQlYUbmd",
|
||||
"outputId": "71217cbf-f7d0-4496-c150-1c363386a16f",
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "stream",
|
||||
"name": "stdout",
|
||||
"text": [
|
||||
"Fri Feb 23 03:18:02 2024 \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
|
||||
"|-----------------------------------------+----------------------+----------------------+\n",
|
||||
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
|
||||
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
|
||||
"| | | MIG M. |\n",
|
||||
"|=========================================+======================+======================|\n",
|
||||
"| 0 Tesla V100-SXM2-16GB Off | 00000000:00:04.0 Off | 0 |\n",
|
||||
"| N/A 33C P0 24W / 300W | 0MiB / 16384MiB | 0% Default |\n",
|
||||
"| | | N/A |\n",
|
||||
"+-----------------------------------------+----------------------+----------------------+\n",
|
||||
" \n",
|
||||
"+---------------------------------------------------------------------------------------+\n",
|
||||
"| Processes: |\n",
|
||||
"| GPU GI CI PID Type Process name GPU Memory |\n",
|
||||
"| ID ID Usage |\n",
|
||||
"|=======================================================================================|\n",
|
||||
"| No running processes found |\n",
|
||||
"+---------------------------------------------------------------------------------------+\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!nvidia-smi"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZI4EwAnEUTFC"
|
||||
},
|
||||
"source": [
|
||||
"## Install Dependencies"
|
||||
]
|
||||
},
|
||||
@@ -23,18 +88,18 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "powershell"
|
||||
}
|
||||
"id": "NaEjHlAWULVd"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install -q inference \"supervision[assets]\""
|
||||
"!pip install -q inference-gpu \"supervision[assets]\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "zUKICBWFULVe"
|
||||
},
|
||||
"source": [
|
||||
"## Download a Video Asset\n",
|
||||
"\n",
|
||||
@@ -44,94 +109,196 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "MyNzGpzOULVe"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"\n",
|
||||
"# Download a supervision video asset \n",
|
||||
"# Download a supervision video asset\n",
|
||||
"path_to_video = download_assets(VideoAssets.PEOPLE_WALKING)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "0cMveUy5U-tN"
|
||||
},
|
||||
"source": [
|
||||
"## Tracking Objects\n",
|
||||
"<div style=\"display: flex; justify-content: center;\">\n",
|
||||
" <video controls width=\"1000\"\">\n",
|
||||
" <source\n",
|
||||
" src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\"\n",
|
||||
" type=\"video/mp4\"\n",
|
||||
" >\n",
|
||||
" </video>\n",
|
||||
"</div>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "lowrxifpULVe"
|
||||
},
|
||||
"source": [
|
||||
"## Tracking Objects in a Frame\n",
|
||||
"\n",
|
||||
"Now that we have our video installed, let's get to work on tracking objects. We'll first pull in a model from roboflow Inference to detect people in our video. Then let's create a `byte_tracker` object that we'll pass our detections to. This will give us a `tracker_id`. We'll then utilize that tracker id to label our detections with a `label_annotator` to display the tracker id. Finally, we'll use a utility called `VideoSink` to save the annotated frames to a video. Let's dive in to the code."
|
||||
"Now that we have our video installed, let's get to work on tracking objects. We'll first pull in a model from roboflow Inference to detect people in our video. Then let's create a `byte_tracker` object that we'll pass our detections to. This will give us a `tracker_id`. We'll then utilize that tracker id to label our detections with a `label_annotator` to display the tracker id."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "gtbnWeT8ULVe",
|
||||
"outputId": "0fbc468d-ea0d-4d23-a44f-1d50e37f95bf"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"output_type": "display_data",
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
],
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9Z5BtyXUeCn6Zex9T59Qpe+uaut63B9qi0bCEIQmSoGhEmSDFCSr0Y2KkCEUoNKEfMz9GPxUx789Ib16E3rwgFCRECSDBAQQSIIAGugl0A93o7tvXtLveVNWtqlveHLv3zvmRe+VemTv3qVNN8s1jRK0bdc8526RZuXK5XLlSKKUU9mAP9mAP9mAP9mAP9mAP9mAP9mAP/p6C/P93A/ZgD/ZgD/ZgD/ZgD/ZgD/ZgD/ZgD/4msGfY7sEe7MEe7MEe7MEe7MEe7MEe7MHfa9gzbPdgD/ZgD/ZgD/ZgD/ZgD/ZgD/bg7zXsGbZ7sAd7sAd7sAd7sAd7sAd7sAd78Pca9gzbPdiDPdiDPdiDPdiDPdiDPdiDPfh7DXuG7R7swR7swR7swR7swR7swR7swR78vYY9w3YP9mAP9mAP9mAP9mAP9mAP9mAP/l7DnmG7B3uwB3uwB3uwB3uwB3uwB3uwB3+vYc+w3YM92IM92IM92IM92IM92IM92IO/1xAO+uC///f/vu99IYT1WQRKKesd+pNSolQqQUqZu9+vnDiOEUUR4jiGUsoqk7/vlkV10rUwDBGGYe4ZX71CCCilzB8ASCnxJ3/yJ7h8+TKefvppHD16FBMTExgaGjL18Hc3NzcRRRE2NzcxOzuL6elpfPSjH0WpVDLPSSktfFD9vN4kSXJ92c048LL4O3TdxSO/F8cxkiSx3k+SBL1eD3EcmzFJkgQXL17E97//fVSrVUxNTeEzn/kMqtUq7t27h4sXL2JxcRFBEEBKacorl8v4xV/8RXz0ox9FFEVWW3i9ABAEAcIwxH/9r/8VV69exfT0NEZGRtBqtRCGISqVCqIoQq/XQ7vdRrvdxtraGoIgwNjYGP71v/7XGB8fR5IkFn24+HdxSM+XSiUEQWC1aaexIBwCGf2VSiXvO0mSIIoiUyene+p/EARWnXxuSSkRBIGhP/4M4dalIV/7Od3w++Vy2fSfnvc96/af44DmP+HU97z7qZRCr9dDkiQIw/7srF97hBCoVCp2X6Gg4G8LBxobl3/46nav8fZXKpXCZ/q1nehACIGhoSHMzc3hj/7oj0y5RJsAMDw8jF//9V/HxMSEt328fKUUyuUy/vRP/xR/+Zd/iVKphC9+8Yv4+Mc/jrGxMfyn//SfcP36dZw5cwb/6l/9K1QqFcOP+vFuF2g+B0Fg6L8I3Pbx79RX3/0iSJIEc3NzWF9fx9LSEu7du4cDBw6gXq+jVCp552GSJNi3bx9OnTpl1RXHMcrlssUz+NgV9Z2A5gHRAR8TzvcIaF7z5zqdDmq1Wk4e83lfRIsA0Ol0jCwedPw4EF+iuej23y3Th99er4dyuVwof11w5T3JnVKphECGECiek27b6HcURUamuM/yOenyKi7r4zhGtVot7OtO13u9HpRShm/z8otw4ZbX7XYNbyT8KCgACi5aeLkcDwDM+67s4Nd4m/i1brdrypBSIhBh7hmCRCU5nhtFkRmPcrlcSAM0HhcuXMB//s//GUmS4NChQ/jN3/xN/P7v/58wMjyC96++hz/6oz/C4cOH0Wg0UKlUcOvWLZw/fx4jIyNIksTMX7cepZSRlcRn+tF2EZBuxPHo1uP+9uHYHQPSx9x7Lg/wlc/bXzTvSOdSShm+49bpvp8sTKH3g08iR2wAwirw5O9JvP3VBHEEPPSrAuuzQFgGTn1W4NX/V4LOpvOSjFH6te8D9a1c++I4hlobBb7/Syhasxs5CmzdB5IIEBIYPQKszwIq9j6u58rnvguMr3p5qg9P9OnjL0Rbg+jnxEv5+/y+awdwXhTHseEf/F6328VLL72Ey5cvG3lCf0VyIY5jnDt3Dv/4H/9jM+6+OSKlzOlALg54Xf1sDyEEwjA0vI/+/i//53/ZF28EAxu2OwE32oqEl+8eMWoS0lxQFxlqHCncCHSVKvd9H2ERAvsNSBHj4QNWqVTwqU99CpcvX8Zrr72GCxcuIAxDDA8PY2hoCKVSCe12G1EUodlsotVq4ejRo3jqqacwOjpqmL7LiHYCjj8X5/0MDFcg+XBV1H9+n48pF77cYBFC4MyZM+h0OhgeHsbY2BiklJibm8Nrr72G7e1tVCoVTExMQEqJxcXFHCP3tZW3q1qtYnl5Gc1m0yilgDb4er2eZdjRmAPA2NgYkiTBwsICJicnrTq4YdtPmHIFqAiKDMYoiizDk3BXJGy4MUhChRusLv37lJVBFP5Bgde5kxLre5f+yPAqErr8OxeyZOS7fGO3CsdOBlW/PpDD4cO+zwXFTs/yT18ZSZKgXC6jXC4jDENLqJHSf/fuXTMH+5VPuO12u6afMzMzUEobvCdOnMCNGzfwwQcf4Oc//zk++9nPmmepPTspbbweer7IscH7yn/TO/2Utn7lcQUiDEN0Oh3jZHF5MuGR172b+gahxZ2M0J3qc/nAoO/Rs3EcWw7WQWib44TkMK+HGwyDlEN8bdDneV30XSlVaNT2K494P3ca+viJDz/8OzlSudzZCVzeT3OBeORORi2/x/th6RUQUKK/s4WeLZVKufnp4sF1Rrp9CILAOBugUmMBeT5d1Bdy2nIHnmuskRwlnWJrawulUgm9Xg9vv/02xsbG8NnPfBY/ffWn+NGPfoRWq4UkUXj66afwpS99CRMTE9Z8LqJ9Vz5zPOw0NnSf6Klo7rq/i57j85zK9+m3O4Hv2UHojNdZpK8MUj8AJD3gxosKhz4qUKoBm/NArz3Qq1Y7CuxTAxv3su8qAdbuDl5HEfC+c5pxaZW3dSeg/gRB0FceUv2ubkS8i5dFjt8XXngBd+7cwdraWs7ALIIDBw6gWq16dZ1+8sb329VJqAxe3t8U/tYM20HAxygI6T5mxZ/h312r3zVCXOBGs1sW9woAxcrKTkw4jmOcP38eH//4x/H973/fXGs2mwiCAIcOHcLMzIwl4Hu9Hk6cOIF2u42RkZFdKdcuM+MM10dggzBM3keXgRdNXp9BTE4CWskC9Orro48+iq2tLSPwh4eHcezYMYRhiJGRETQaDSwtLWF5edliCjTWvC4Slrze69evY21tzWpPuVxGpVJBu922lBZS/icnJ9Hr9XD9+nU8+uijlkeK00uRYsKFidvGIseAOwaVSsUYtW45RULNZV68PbxsPidIOXHLct/zldWP/l3j38VD0Xyi73yVw21D0TXCATHIQZVm3/zgePLNf7cPrhAZpP4i/NH7gyjxVGeRUkXOlXq9bpxIvV4PnU7HPBPHMe7cuYOHH34Y1Wo1V5Zr1G5ubmJpackoyLQCBGhnEtHg1atX8fzzz5s27jRvfHggA7Pfir1LY1yJAPJ49PE4DlEUodvtot1uQ0qJkZERs8Lkg53mJefBRX3YCR+uYeB+FvEVusdx08/w89UL2LQ8qJLB5xVfte1n+BXxCDKCuF4wCC/1tSMQGS0NomxzY41kV1EkC3fG8X6RDCHHkutg4nX5jCL+1+v1AGj5WdTnIiAc+nhLPz7HcU5ylvQYeoa+k0whxdmnZ3Deqpx63H7oa3mdguOG2sbLDYIAt27dwv/0//yfMD8/DwV9f3NzEzMzM/jBD36AS5cuYWhoCE8//TRefvllrK2t4ZVXXsH58+dx5MgR45Tj/XdlIBmlPkfNoHOFynFXO32yrgg4nwZgjY0Pd279RfPffcedez590G2Txfv7dgKIu9A0IYB954DutkKpJrB8XSHpFXbeW5+UEiIMEQkBDDYUu4J+/NQH3CHWz+DrVw7xHuMYKmgX0STnGe68Il1aCIHJyUlsb2+j1Wp528DbJ4XE5OSkVy776nDv82u0gEfRZFxm8XoH0Rn6wd+6YetrkKvo8AnEkVHk1fUpkm59RUySGzMu06aVWrceX5+IKIqAPLOf+MQncOHCBWxtbWWhN0GAz33uc3j77bfx9ttvG6bWbrcxPDyMffv2FdbZz+AAUCi4fIKSl+MSk0/JKDJsuIBxjRJu6JHXP45jayWEv/f000+b93u9HtbX1y3jktrGV7QJyBisVqt499138eKLL2J9fR2NRsO0kZQKCpMkBVgpbVBKKVGr1fDBBx9gfX0dExMTxvtbhA+Ox34GXNFverfX66FSqVhhYr73XLzzfrirSu5Y8SgGHtXAx9UXQl1k8Pj6zqMd+inMPuDMjfexHxPn313PH2+zD3zjNahC4iuL6Nt1SvmUB9+1onnkA+qvS4dCCOO4qVarqNVqGBkZwczMDCYnJ40gofDO1dVV3Lt3D+fPn/cqo2EYYn19Ha+//jpefullrKyu4MCBA1hZWcHExATGxsawvb2Ne/fumXdnZmawsrKCWq2WcyT6okl4+10F0tdvl4e5NMPpaCelgd8n51u320Wv10Oj0cDa2hqq1WqOB/P32+224Wv8ej/6L1IyeZ9obhfxFRdPPlxyXPmUB9dAcMvqp0QVAcc9yTfOk3xztWiOlkolY2TQc7tRdOjZKIoQyhBA8ep10TWSOUX6hY8muLFFny4eOAxC96TUUpmurHXnj/s+kCnYtPpD5uUgPJ7rZDyk+f79+7h48SLm5+dx4MABfOYzn8HQ0FCOP3H8JUkClSSAE51j0TCyVXZXZ+ErnRyntPXpD//wD3F//r4pq9lsIgxDnDhxAnEco9VqodPp4K233sLGxgaCIECr1cJ3v/tdPP3000ZHGBQGjULwQT9HwCC/fePFcd1vju0kY/h7g+o0bhnG6VBYExB1gQ++myDq6AeXrwMHHgeaSwrzl/u82Ke9fzcwWNmuXePTZTjfLZJT7n36TQ5XdzsCpyNyoPn0VLine truncated
|
||||
},
|
||||
"metadata": {}
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import supervision as sv\n",
|
||||
"from inference.models.utils import get_roboflow_model\n",
|
||||
"\n",
|
||||
"# Load a pre trained yolov8 nano model from Roboflow Inference.\n",
|
||||
"model = get_roboflow_model('yolov8n-640')\n",
|
||||
"\n",
|
||||
"# Create a video info object from the video path.\n",
|
||||
"video_info = sv.VideoInfo.from_video_path(path_to_video)\n",
|
||||
"\n",
|
||||
"# Create a label annotator for labeling detections with our tracker_id.\n",
|
||||
"label = sv.LabelAnnotator()\n",
|
||||
"\n",
|
||||
"# Create a ByteTrack object to track detections.\n",
|
||||
"byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
|
||||
"\n",
|
||||
"# Create a frame generator from video path for iteration of frames.\n",
|
||||
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
|
||||
"\n",
|
||||
"# Grab a frame from the frame_generator.\n",
|
||||
"frame = next(frame_generator)\n",
|
||||
"\n",
|
||||
"# Run inference on the frame by passing it to our model.\n",
|
||||
"result = model.infer(frame)[0]\n",
|
||||
"\n",
|
||||
"# Convert model results to a supervision detection object.\n",
|
||||
"detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
"# Update detections with tracker ids fro byte_tracker.\n",
|
||||
"tracked_detections = byte_tracker.update_with_detections(detections)\n",
|
||||
"\n",
|
||||
"# Create labels with tracker_id for label annotator.\n",
|
||||
"labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
|
||||
"\n",
|
||||
"# Apply label annotator to frame.\n",
|
||||
"annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
|
||||
"\n",
|
||||
"# Display the frame.\n",
|
||||
"sv.plot_image(annotated_frame)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "wR2nrNUiVOGq"
|
||||
},
|
||||
"source": [
|
||||
"## Tracking Objects in a Video\n",
|
||||
"\n",
|
||||
"Finally, we'll use a utility called `VideoSink` to save the annotated frames to a video. Let's dive in to the code."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "b4lOhjcfVYAl"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import supervision as sv\n",
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"from inference.models.utils import get_roboflow_model\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"# Load a pre trained yolov8 nano model from Roboflow Inference.\n",
|
||||
"model = get_roboflow_model('yolov8n-640')\n",
|
||||
"\n",
|
||||
"if __name__ == '__main__':\n",
|
||||
"# Create a video info object from the video path.\n",
|
||||
"video_info = sv.VideoInfo.from_video_path(path_to_video)\n",
|
||||
"\n",
|
||||
" # Install our video from supervision assets.\n",
|
||||
" PATH_TO_VIDEO = download_assets(VideoAssets.PEOPLE_WALKING)\n",
|
||||
"# Create a label annotator for labeling detections with our tracker_id.\n",
|
||||
"label = sv.LabelAnnotator()\n",
|
||||
"\n",
|
||||
" # Load a pre trained yolov8 nano model from Roboflow Inference.\n",
|
||||
" model = get_roboflow_model('yolov8n-640')\n",
|
||||
"# Create a ByteTrack object to track detections.\n",
|
||||
"byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
|
||||
"\n",
|
||||
" # Create a video info object from the video path.\n",
|
||||
" video_info = sv.VideoInfo.from_video_path(PATH_TO_VIDEO)\n",
|
||||
"# Create a frame generator from video path for iteration of frames.\n",
|
||||
"frame_generator = sv.get_video_frames_generator(path_to_video)\n",
|
||||
"\n",
|
||||
" # Create a label annotator for labeling detections with our tracker_id.\n",
|
||||
" label = sv.LabelAnnotator()\n",
|
||||
"# Create a video sink context manager to save resulting video.\n",
|
||||
"with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
|
||||
"\n",
|
||||
" # Create a ByteTrack object to track detections.\n",
|
||||
" byte_tracker = sv.ByteTrack(frame_rate=video_info.fps)\n",
|
||||
" # Iterate through frames yielded from the frame_generator.\n",
|
||||
" for frame in tqdm(frame_generator, total=video_info.total_frames):\n",
|
||||
"\n",
|
||||
" # Create a frame generator from video path for iteration of frames.\n",
|
||||
" frame_generator = sv.get_video_frames_generator(PATH_TO_VIDEO)\n",
|
||||
" # Run inference on the frame by passing it to our model.\n",
|
||||
" result = model.infer(frame)[0]\n",
|
||||
"\n",
|
||||
" # Create a video sink context manager to save resulting video.\n",
|
||||
" with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
|
||||
" # Convert model results to a supervision detection object.\n",
|
||||
" detections = sv.Detections.from_inference(result)\n",
|
||||
"\n",
|
||||
" # Iterate over frames of the video.\n",
|
||||
" for frame in frame_generator:\n",
|
||||
" # Update detections with tracker ids fro byte_tracker.\n",
|
||||
" tracked_detections = byte_tracker.update_with_detections(detections)\n",
|
||||
"\n",
|
||||
" # Run inference on the frame by passing it to our model.\n",
|
||||
" result = model.infer(frame)[0]\n",
|
||||
" # Create labels with tracker_id for label annotator.\n",
|
||||
" labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
|
||||
"\n",
|
||||
" # Convert model results to a supervision detection object.\n",
|
||||
" detections = sv.Detections.from_inference(result)\n",
|
||||
" # Apply label annotator to frame.\n",
|
||||
" annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
|
||||
"\n",
|
||||
" # Update detections with tracker ids fro byte_tracker.\n",
|
||||
" tracked_detections = byte_tracker.update_with_detections(detections)\n",
|
||||
"\n",
|
||||
" # Create labels with tracker_id for label annotator.\n",
|
||||
" labels = [ f\"{tracker_id}\" for tracker_id in tracked_detections.tracker_id ]\n",
|
||||
"\n",
|
||||
" # Apply label annotator to frame.\n",
|
||||
" annotated_frame = label.annotate(scene=frame.copy(), detections=tracked_detections, labels=labels)\n",
|
||||
"\n",
|
||||
" # Save the annotated frame to an output video.\n",
|
||||
" sink.write_frame(frame=annotated_frame)"
|
||||
" # Save the annotated frame to an output video.\n",
|
||||
" sink.write_frame(frame=annotated_frame)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"id": "0J10FczGULVe"
|
||||
},
|
||||
"source": [
|
||||
"# View the Video\n",
|
||||
"Let's take a look at our resulting video. It will also be created in your current directory with the name `output.mp4` Notice how even with a little flicker, we can see the `tracker_id` on the people walking in the video. With trackers under your belt, there are now a wide variety of use cases you can solve for! Happy building!\n",
|
||||
"\n",
|
||||
"Let's take a look at our resulting video. It will be created in your current directory with the name `output.mp4` Notice how even with a little flicker, we can see the `tracker_id` on the people walking in the video. With trackers under your belt, there are now a wide variety of use cases you can solve for! Happy building! "
|
||||
"<div style=\"display: flex; justify-content: center;\">\n",
|
||||
" <video controls width=\"1000\">\n",
|
||||
" <source\n",
|
||||
" src=\"https://media.roboflow.com/supervision/cookbooks/object-tracking.mp4\"\n",
|
||||
" type=\"video/mp4\"\n",
|
||||
" >\n",
|
||||
" </video>\n",
|
||||
"</div>"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "V100",
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "venv",
|
||||
"language": "python",
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
@@ -144,9 +311,9 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.7"
|
||||
"version": "3.11.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
+1306
File diff suppressed because it is too large.
Load diff
Vendored
+376
-253
File diff suppressed because it is too large.
Load diff
+745
@@ -0,0 +1,745 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RFjrV07Spmm2"
|
||||
},
|
||||
"source": [
|
||||
"# Serialise Detections to a CSV File\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"[](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/detections-to-jsonsink.ipynb)\n",
|
||||
"\n",
|
||||
"This cookbook introduce [sv.CSVSink](https://supervision.roboflow.com/latest/detection/tools/save_detections/#supervision.detection.tools.csv_sink.CSVSink) tool designed to write captured object detection data to file from video streams/file"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aFQHoaDLp8R3"
|
||||
},
|
||||
"source": [
|
||||
"Click the `Open in Colab` button to run the cookbook on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "hzMGlRKlel4p"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install -q inference requests tqdm supervision==0.21.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"id": "6vRfXc_Je5Ee"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import csv\n",
|
||||
"from typing import List\n",
|
||||
"from collections import defaultdict\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"import supervision as sv\n",
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"from inference import InferencePipeline\n",
|
||||
"from inference.core.interfaces.camera.entities import VideoFrame\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "d4s3Ndsowl81"
|
||||
},
|
||||
"source": [
|
||||
"The parameters defined below are:\n",
|
||||
"* `SOURCE_VIDEO_PATH` - the path to the input video\n",
|
||||
"* `CONFIDENCE_THRESHOLD` - do not include detections below this confidence level\n",
|
||||
"* `IOU_THRESHOLD` - discard detections that overlap with others by more than this [IOU](https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy/) ratio\n",
|
||||
"* `FILE_NAME` - write the json output to this file\n",
|
||||
"* `INFERENCE_MODEL` - model id. This cookbook uses a [model alias](https://inference.roboflow.com/quickstart/aliases/), but it can also be a [fine-tuned model](https://inference.roboflow.com/quickstart/explore_models/) or a model from the [Universe](https://inference.roboflow.com/quickstart/load_from_universe/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"id": "urRjZjh2f30v"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"people-walking.mp4 asset download complete. \n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"SOURCE_VIDEO_PATH = download_assets(VideoAssets.PEOPLE_WALKING)\n",
|
||||
"CONFIDENCE_THRESHOLD = 0.3\n",
|
||||
"IOU_THRESHOLD = 0.7\n",
|
||||
"FILE_NAME = \"detections.csv\"\n",
|
||||
"INFERENCE_MODEL = \"yolov8n-640\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "XQnVdIEcjVhy"
|
||||
},
|
||||
"source": [
|
||||
"As a result of executing the above `download_assets(VideoAssets.PEOPLE_WALKING)` , you will download a video file and save it at the `SOURCE_VIDEO_PATH`. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
|
||||
"\n",
|
||||
"<video controls width=\"1280\">\n",
|
||||
" <source src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\" type=\"video/mp4\">\n",
|
||||
"</video>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "nf3sxLWfwrGL"
|
||||
},
|
||||
"source": [
|
||||
"## Read single frame from video\n",
|
||||
"\n",
|
||||
"The [`get_video_frames_generator`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.get_video_frames_generator) enables us to easily iterate over video frames. Let's create a video generator for our sample input file and display its first frame on the screen.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "BG8Dqn1WwsPa",
|
||||
"outputId": "46d4ada1-4e61-4b47-fca4-352b013cd8d1"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9aYxm13UeCj/7DO9U89DzzG52N4emJJKiSM2D6UGwEg8IktzERoxc4PvhAAECBMiPAPkd4Lt/vuT+8UWundgZbMWyJVmWYFoWZYqSSIrsJrubYpPdzZ6qu6qrqmt+x3PO/n7ss9ZZe5993qpm7ocPBmo3GlX1vufsYe2117zXUlprjd2223bbbtttu2237bbdttt2227bbbvt72gL/v89gd2223bbbtttu2237bbdttt2227bbbvtf6XtKra7bbfttt2223bbbtttu2237bbdttv+TrddxXa37bbdttt2227bbbttt+223bbbdtvf6bar2O623bbbdttu2227bbfttt2223bbbvs73XYV292223bbbtttu2237bbdttt2227bbX+n265iu9t2227bbbttt+223bbbdttu22277e9021Vsd9tu2227bbfttt2223bbbtttu223/Z1uu4rtbtttu2237bbdttt2227bbbttt+22v9NtV7Hdbbttt+223bbbdttu2227bbfttt32d7pFO33w3/27fzf0e6UU/x/WtNbQWlvPK6UQBAGiKEIQFLp2VZ/UBwBkWYYkSZCmaalf6gMAgiCw+nHHj6IIURSVnvHNXynFc8iyjPv4H//jf+DixYt45plncOTIEUxNTaHVavHYNGetNTY3N5EkCTY2NjA3N4eDBw/i4x//OKKo2JIwDC14yLVnWcZ/07Pbzd3th+BHv8t35D5J2NHnNAf3/SzLMBgMkCQJf5dlGS5cuIC//uu/Rr1ex969e/GFL3wBjUYDt27dwttvv4379+/z/lN/cRzjl37pl/Cxj30MaZpa+ypxgNYfhiH++3//73j//fdx8OBBjI+Po9PpIIoi1Ot1JEmCwWCATqeDXq+HlZUVRFGEyclJ/Mt/+S8xNTXFc1ZKMUyrYCn3IY5jhGFozWm7vdBaI01TAEAURQjDELVazfsO4TmNmaYpkiRBkiQ81zAMuV/aM9q/IAh4PXK/lFLch3tGfPOXOCy/j+MYURR51+/7jD6XMIjj2JpfVT/yp9Yag8EAWZZZZ2cn77ut0WjYa4UGFMz/IS3NzF7EUTx0r32fZbqgX/V6ffictb8fel8phWazibv37uIP/8sfoj/oQ2caURwhCg1sRkZH8Pf/3t/H9PR09Tjis7gW40//55/iL//yLxHHMV588UW88MILmJycxH/4j/8B165ew6lTp/Av/sW/QK1es+hhoHZoN1WAzjTCMEQcx0MfdedHLdPm3MZR7P2+qmVZhrm7c1hfW8fS0hJu3b6Fffv2YXRkFFEclfZTZ+b8zczO4JFHHrHGytIMtXrNotk600PnIftP0gQAGA/c99yzoZRCGBiao2HG6ff6aLVaNv9zUJJwqfQZgF6vhyAI+CxCofTssEZ0KYxCKCjeF996fX9nWYZkkBg66DlKVfQ1UAW9S9MUmc4QRzFChJVzHbYvg8GAZQKal4t7kv5Tk7w+TVM0Gg1+3uXlcv5VcyC65pNjquYv++v3+1Yf/F2AEnxLPB4aaZJCQzNtc3mHi5MWbYaGgkK/34eGNjxOhQhyf4qXJkJDK/vzQWLkiSiMENdiLz7SPLIsw1vn38L/9Xv/F7Isw4EDB/Drv/7r+O3f+m2Mj43jvSvv4b/80X/B4UOHMTY2hnq9jg8//BCnz5zGxMQEsjTjfcq0vbdaG3hAgemUi9sKDv/wtCzLTD/0TqDoBS9cpKxD55zeJbzXWiNJE+hMyHJK7BOKvfKti9+pWAfjRE7PgjBgeKRZWnqfYeIgmZy/hAc0oAJDz9w+JH1z5yz/ztIMmTYyaRAEhufl9KuKBrlypEuvZOMzmPO1Kv1A9uOj11Xvuv0kacJ4UjVv2sssy3jvAcNLkoGRF9Ms5e96/R5++PIPcfHiRea7WWbgJt9353H69Gn8w3/4D1k+pzNiyeBBaOi+Ky+KbklnYDjTd4p+5PREAXEUIwgDi9b87v/rd4fCjdqOFdvt2k6Yl1Q85XfbCd6+Pl3FzjqUvgPmKLLyM59SK5HSp1i7PxuNBj73uc/h4sWL+OlPf4rz588jiiKMjo6i2WwiiiL0ej0kSYJ2u41Op4MjR47g6aefxsTEhFeJGob8BCcJP/d7lwG6e1Qiyg7cpHJb1a/7HiF9EAQW0z916hR6vR5GR0cxNTWFIAhw9+5dvP7669ja2kKj0cDU1BTCMMT9+/etcX0EwoVPo9HA8vIy2u02RkZGmPGEYYjBYMD7LPccACYnJ5FlGRYWFjAzM2ONsZ0QIRXH7ZSqKoUxSRKGl1TsfXgtYZGmqRHgssxSaAFYSq0Phj6msB2hrWoSB6vOzDBlj/67a5DPuL+TACOVfGkQ8eH2do0MCpXN7UIIIZL5PGwjYaFK6HWflT+tPnJhI8sy1Go1xDVjaJBMTSmFLM1w89ZNTE5O+o1mQoDIsgxhGqLX7xkGlyS4fec2ntfPI67FOH78OK5fu44rV67g9Tdex5e++CV+VmttBDZlCzdeYTYzzDMIAqjAzNG7fpqbFkKjBqAK4agkNGl/X0DBRLM0Nxrlwne/10c/7hd0VVXgcz52aX5D2k5wcZhxxzd/V4Am2CuUlVr5jKdDQBnlNI5j0w92OBdBZ9IsRaQiM744J1Vrl/gRqMAoVFnKNIG+9ykKUggvnYuH0ciddYRhyAY3mof7HK2pSuAmQ6qkbTsRaH00lGjkdkqt/I7osmsU3s5QJ58N4sCisb5zoAJlCcWW8qRyWGbG8aCVvb7tcEtrjSiMeD/ctShlcGyQDBAGIcsUG5sbqMU1DAYDXLhwAZOTk/jiF76IH//kx3j55ZfRbrehM42nn3kaX/2Vr2JmegaZzixc8s0vCAOzVkGDpOIo4e82VogCBR3qSkXC/bvqjAM5fctprFKKYzCrlFRf8z27Ezyj51zeK2FReh6qtBYVKms99Ix8dqfzyFKjoGXIoIOCfv2vyDjbNVdfIbqZpmmJd9Bz2zWGawWelMaX8kiWWjyU8CLLMsRxjE9/+tO4efMmVldXvQp4aS5Q2L9vP5qNJgbJoFJ+dPUXGtvab2UcAVmW2XiSP5IhN3JsR6S2af+PKbY7aT5C4SoQvgPlElMSalmgG6KASMVPAp/eC8PQUmp9B6CKyFmWnizDmTNn8MILL+Cll14CYASEdruNMAxx4MAB3Llzh61JgLHIHj9+HJ1OBxMTE5a3Yjvkl2vxrd+FWRXBdPuTypFPGXKfc8chBY2eSRLjhajVanjiiSewtbWFwWCAwWCAkZERHD16FFEUYXx8HGNjY1haWsLy8jIzMIKtuy9ScabPr169itXVVWs+tVoN9XodnU6H4UQCQr1ex8zMDAaDAT744AM88cQTvG4J32FKGjF8gpErjPjekd9nWYZGo1FSaocxFhI0SPBycVrumVReJExdGLpzlvOuwn/ZT9Vct2vy/A4TFOVP+k9wkHMY1uQY7j5V7aF81zcn11q93djW2pFbwKHZo7pd851b+kkW6larxUakJEnQ6/agoRGqEEma4NbNW3jsscfQbDTtOeXWbcAom2maYnljGUtLS1CBYdSD/gBplkJBodloMvzff/99vPDCCyX8VWpnnluOQhgktjLK+qvAA1JgxXwZD8LyWFW8RMPQp36/j263izAIMTExgV6vV60UER/WhUdKzlVnRnhnL0w+jisw+/qVz/j4mLsmOQYpsQx/IRz64OA9q9r8tzzOOzzT8lxlqdnLMAwRqACpTq3nfOuRf2dphoE2igoJhrSm7RR1hq82/WRBVhKSdnJWgyBg3iX5rI+2Vu1VGIZseNS6iExxn5P0SMo2gJERAL/hbSdKoc/oDaCMu3JOJIzrQsHfam+hHtYL+OTnLEszqEAxL+H9JaVEG0UuQNnwWSnrKc98UMCH8IG8X5k2ML7x4Q38H//v/wP3Fu4BMB6kjY0N3L5zG3/913+Nd955B81mE888/Qxe/uHLWFtdw6uvvoozZ87g8OHDxruti4gTlw8HKjCKudL8HK1VrsNrgBGLYiUs9y4is/nDw5w5opW0x3JOPkWDnvGN4eONhAfuuizjCQKjSDpn2ze2l/5JAyEZDPKfw+Tu0niBQqQig3NiD13a8FHkFHpf8rLt+iGvtpwjNZ884TYF4+HNVGYU9ornpDyUaRPxwp/n55zOUZIlgAKmZ6axtbWFdqftNV66ODAzM1OSFd25+/DN3W+ScweDgVHYla2YkyL8v2qE+H9csd0OmV3h0iVwLvEDbO9s1QZUEUn63KfY+pTaqjUppbwbS20wGCAIAnzmM5/B+fPnsbGxwZsYhiG+/OUv48KFC7hw4QIzgW63i9HR0ZKnUI5ZSfzzJhmXC2cfTH0KgjuOSySrlGKfgiAFALL6A2CPNYW4BUGARqOBZ555hvdjMBhgbW3NWjutTxonaCwK1W40Grh8+TK+//3vY21tDWNjYzxHEioajQYLzzSner0OpUz45vvvv4+1tTVMT0+zF1Wuz7dWqZS5+zLsbym01Ov1Iuyv4j13z6RC586R9oAEACs01BFwCJ99IcQuvvmEYhp3J2eoqsl5SsVenrVhyq5cn2xVhNEH54/K6Ijpk4fL/c5Hqyy4QrGA5F5X8DUShKy15YJjp92BUgr1sI6R1gjGx8cxd2cOMzMzCIIA/UEfYc0YdFZWVnDn9h2cOXOmJIwCZj/X1tbw+uuv4+WXX8bKygr27duHBw8eYHp6GlOTU9jc2sTt27f53bk7c1heXkar1UIYmHOZBRkCZX6W4EbTp3OkgoLJ5esyP/K9zoTnILine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH)\n",
|
||||
"frame = next(generator)\n",
|
||||
"\n",
|
||||
"sv.plot_image(frame, (12, 12))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2YyE30JOS4Sf"
|
||||
},
|
||||
"source": [
|
||||
"## Initialize ByteTrack\n",
|
||||
"\n",
|
||||
"[ByteTrack](https://supervision.roboflow.com/latest/trackers/#bytetrack) is a multi-object tracking algorithm used by Supervision to track and link detected objects across multiple frames, providing consistent IDs for each object.Initialize the object."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"id": "Jpw_TzAsm2oL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"byte_track = sv.ByteTrack(minimum_consecutive_frames=3)\n",
|
||||
"byte_track.reset()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "IjYwj2vbTLBd"
|
||||
},
|
||||
"source": [
|
||||
"## Initialize CSVSink\n",
|
||||
"\n",
|
||||
"To save detections to a `CSV` file, open our [`sv.CSVSink`](https://supervision.roboflow.com/latest/how_to/save_detections/#save-detections-as-csv) and then pass the [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) object resulting from the inference to it.\n",
|
||||
"\n",
|
||||
"Note that empty detections will be skipped."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"metadata": {
|
||||
"id": "-yvv6N1uTKSl"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"csv_sink = sv.CSVSink(FILE_NAME)\n",
|
||||
"csv_sink.open()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-YvmJW5yud1G"
|
||||
},
|
||||
"source": [
|
||||
"## Process video and save detections to csv file\n",
|
||||
"\n",
|
||||
"The [`InferencePipeline`](https://inference.roboflow.com/using_inference/inference_pipeline/) interface is made for streaming and is likely the best route to go for real time use cases. It is an asynchronous interface that can consume many different video sources including local devices (like webcams), RTSP video streams, video files, etc. With this interface, you define the source of a video stream and sinks.\n",
|
||||
"\n",
|
||||
"All the operations we plan to perform for each frame of our video - detection, tracking, annotation, and write to csv are encapsulated in a function named `callback`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"metadata": {
|
||||
"id": "eRQiVkblapCk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def callback(predictions: dict, frame: VideoFrame) -> np.ndarray:\n",
|
||||
" detections = sv.Detections.from_inference(predictions)\n",
|
||||
"\n",
|
||||
" # Only keep person detections\n",
|
||||
" detections = detections[detections.class_id == 0]\n",
|
||||
" detections.data[\"class_name\"] = np.array([\"person\" for _ in range(len(detections))])\n",
|
||||
"\n",
|
||||
" detections = byte_track.update_with_detections(detections)\n",
|
||||
" csv_sink.append(detections, custom_data={'frame_number': frame.frame_id})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "3GRH_QdHgGiC",
|
||||
"outputId": "2baa2f16-29e7-49a1-b2ce-5dc26397c189"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n",
|
||||
"UserWarning: Specified provider 'OpenVINOExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"pipeline = InferencePipeline.init(\n",
|
||||
" model_id=INFERENCE_MODEL,\n",
|
||||
" video_reference=SOURCE_VIDEO_PATH,\n",
|
||||
" on_prediction=callback,\n",
|
||||
" iou_threshold=IOU_THRESHOLD,\n",
|
||||
" confidence=CONFIDENCE_THRESHOLD,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 35,
|
||||
"metadata": {
|
||||
"id": "bYpA9VxBsOvS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pipeline.start()\n",
|
||||
"pipeline.join()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 36,
|
||||
"metadata": {
|
||||
"id": "RrewOF02AYta"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"csv_sink.close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "u7Akx7aUsh75"
|
||||
},
|
||||
"source": [
|
||||
"## Visualizate results of detections csv data with Pandas\n",
|
||||
"\n",
|
||||
"Let's take a look at our resulting data with by using Pandas.\n",
|
||||
"\n",
|
||||
"It will also be created in your current directory with the name detections.csv as well."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 37,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 424
|
||||
},
|
||||
"id": "Bu7AZ3QHiAqb",
|
||||
"outputId": "a0da3923-b9b7-4737-c8df-a063d2d8453d"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>x_min</th>\n",
|
||||
" <th>y_min</th>\n",
|
||||
" <th>x_max</th>\n",
|
||||
" <th>y_max</th>\n",
|
||||
" <th>class_id</th>\n",
|
||||
" <th>confidence</th>\n",
|
||||
" <th>tracker_id</th>\n",
|
||||
" <th>class_name</th>\n",
|
||||
" <th>frame_number</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>1142.0</td>\n",
|
||||
" <td>950.0</td>\n",
|
||||
" <td>1245.0</td>\n",
|
||||
" <td>1080.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.767850</td>\n",
|
||||
" <td>185</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>750.0</td>\n",
|
||||
" <td>450.0</td>\n",
|
||||
" <td>823.0</td>\n",
|
||||
" <td>620.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.748268</td>\n",
|
||||
" <td>168</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>1419.0</td>\n",
|
||||
" <td>702.0</td>\n",
|
||||
" <td>1526.0</td>\n",
|
||||
" <td>887.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.550286</td>\n",
|
||||
" <td>179</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>1674.0</td>\n",
|
||||
" <td>12.0</td>\n",
|
||||
" <td>1727.0</td>\n",
|
||||
" <td>141.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.546864</td>\n",
|
||||
" <td>174</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>1616.0</td>\n",
|
||||
" <td>18.0</td>\n",
|
||||
" <td>1655.0</td>\n",
|
||||
" <td>127.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.520566</td>\n",
|
||||
" <td>181</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>...</th>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8757</th>\n",
|
||||
" <td>394.0</td>\n",
|
||||
" <td>471.0</td>\n",
|
||||
" <td>460.0</td>\n",
|
||||
" <td>633.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.466400</td>\n",
|
||||
" <td>244</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8758</th>\n",
|
||||
" <td>115.0</td>\n",
|
||||
" <td>256.0</td>\n",
|
||||
" <td>199.0</td>\n",
|
||||
" <td>399.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.431374</td>\n",
|
||||
" <td>223</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8759</th>\n",
|
||||
" <td>77.0</td>\n",
|
||||
" <td>700.0</td>\n",
|
||||
" <td>169.0</td>\n",
|
||||
" <td>892.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.416318</td>\n",
|
||||
" <td>240</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8760</th>\n",
|
||||
" <td>1010.0</td>\n",
|
||||
" <td>59.0</td>\n",
|
||||
" <td>1055.0</td>\n",
|
||||
" <td>174.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.403422</td>\n",
|
||||
" <td>246</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8761</th>\n",
|
||||
" <td>1758.0</td>\n",
|
||||
" <td>0.0</td>\n",
|
||||
" <td>1809.0</td>\n",
|
||||
" <td>96.0</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.313793</td>\n",
|
||||
" <td>236</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"<p>8762 rows \u00d7 9 columns</p>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" x_min y_min x_max y_max class_id confidence tracker_id \\\n",
|
||||
"0 1142.0 950.0 1245.0 1080.0 0 0.767850 185 \n",
|
||||
"1 750.0 450.0 823.0 620.0 0 0.748268 168 \n",
|
||||
"2 1419.0 702.0 1526.0 887.0 0 0.550286 179 \n",
|
||||
"3 1674.0 12.0 1727.0 141.0 0 0.546864 174 \n",
|
||||
"4 1616.0 18.0 1655.0 127.0 0 0.520566 181 \n",
|
||||
"... ... ... ... ... ... ... ... \n",
|
||||
"8757 394.0 471.0 460.0 633.0 0 0.466400 244 \n",
|
||||
"8758 115.0 256.0 199.0 399.0 0 0.431374 223 \n",
|
||||
"8759 77.0 700.0 169.0 892.0 0 0.416318 240 \n",
|
||||
"8760 1010.0 59.0 1055.0 174.0 0 0.403422 246 \n",
|
||||
"8761 1758.0 0.0 1809.0 96.0 0 0.313793 236 \n",
|
||||
"\n",
|
||||
" class_name frame_number \n",
|
||||
"0 person 1 \n",
|
||||
"1 person 1 \n",
|
||||
"2 person 1 \n",
|
||||
"3 person 1 \n",
|
||||
"4 person 1 \n",
|
||||
"... ... ... \n",
|
||||
"8757 person 341 \n",
|
||||
"8758 person 341 \n",
|
||||
"8759 person 341 \n",
|
||||
"8760 person 341 \n",
|
||||
"8761 person 341 \n",
|
||||
"\n",
|
||||
"[8762 rows x 9 columns]"
|
||||
]
|
||||
},
|
||||
"execution_count": 37,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"df = pd.read_csv(FILE_NAME)\n",
|
||||
"df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "pmGzIUqRkpqI"
|
||||
},
|
||||
"source": [
|
||||
"## Convert CSV data to sv.Detections"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 38,
|
||||
"metadata": {
|
||||
"id": "aa24sW8FI-xV"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def csv_to_detections(csv_file: str) -> List[sv.Detections]:\n",
|
||||
" rows_by_frame_number = defaultdict(list)\n",
|
||||
" with open(csv_file, 'r') as f:\n",
|
||||
" reader = csv.DictReader(f)\n",
|
||||
" for row in reader:\n",
|
||||
" frame_number = int(row[\"frame_number\"])\n",
|
||||
" rows_by_frame_number[frame_number].append(row)\n",
|
||||
"\n",
|
||||
" detections_list = []\n",
|
||||
" for frame_number, rows in rows_by_frame_number.items():\n",
|
||||
" xyxy = []\n",
|
||||
" class_id = []\n",
|
||||
" confidence = []\n",
|
||||
" tracker_id = []\n",
|
||||
" custom_data = defaultdict(list)\n",
|
||||
"\n",
|
||||
" for row in rows:\n",
|
||||
" xyxy.append([row[key] for key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\"]])\n",
|
||||
" class_id.append(row[\"class_id\"])\n",
|
||||
" confidence.append(row[\"confidence\"])\n",
|
||||
" tracker_id.append(row[\"tracker_id\"])\n",
|
||||
"\n",
|
||||
" for custom_key in row.keys():\n",
|
||||
" if custom_key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\", \"class_id\", \"confidence\", \"tracker_id\"]:\n",
|
||||
" continue\n",
|
||||
" custom_data[custom_key].append(row[custom_key])\n",
|
||||
"\n",
|
||||
" if all([val == \"\" for val in class_id]):\n",
|
||||
" class_id = None\n",
|
||||
" if all([val == \"\" for val in confidence]):\n",
|
||||
" confidence = None\n",
|
||||
" if all([val == \"\" for val in tracker_id]):\n",
|
||||
" tracker_id = None\n",
|
||||
"\n",
|
||||
" detections_list.append(\n",
|
||||
" sv.Detections(\n",
|
||||
" xyxy=np.array(xyxy, dtype=np.float32),\n",
|
||||
" class_id=np.array(class_id, dtype=int),\n",
|
||||
" confidence=np.array(confidence, dtype=np.float32),\n",
|
||||
" tracker_id=np.array(tracker_id, dtype=int),\n",
|
||||
" data=dict(custom_data)\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" return detections_list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 39,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "EREtElrA0u49",
|
||||
"outputId": "c83a5658-fb28-4734-a09c-fd94a9e1b45a"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Detections: 341\n",
|
||||
"Detections(xyxy=array([[1142., 950., 1245., 1080.],\n",
|
||||
" [ 750., 450., 823., 620.],\n",
|
||||
" [1419., 702., 1526., 887.],\n",
|
||||
" [1674., 12., 1727., 141.],\n",
|
||||
" [1616., 18., 1655., 127.],\n",
|
||||
" [1388., 2., 1437., 184.]], dtype=float32), mask=None, confidence=array([0.7678498 , 0.7482683 , 0.5502863 , 0.5468636 , 0.5205659 ,\n",
|
||||
" 0.31684005], dtype=float32), class_id=array([0, 0, 0, 0, 0, 0]), tracker_id=array([185, 168, 179, 174, 181, 28]), data={'class_name': ['person', 'person', 'person', 'person', 'person', 'person'], 'frame_number': ['1', '1', '1', '1', '1', '1']})\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"detections_list = csv_to_detections(FILE_NAME)\n",
|
||||
"detections_list\n",
|
||||
"\n",
|
||||
"print(f\"Detections: {len(detections_list)}\")\n",
|
||||
"print(detections_list[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2pyVFVWI096b"
|
||||
},
|
||||
"source": [
|
||||
"###\u00a0Annotate First Frame\n",
|
||||
"\n",
|
||||
"Visualize the first frame of a video alongside the initial detections obtained by parsing CSV data into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) objects. The annotated image will show the original video frame, marked with the first bounding box detected from the parsed data, providing a visual representation of the identified object(s) in the scene."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "qXtulNDT1ChB"
|
||||
},
|
||||
"source": [
|
||||
"### Get back `sv.Detections`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 40,
|
||||
"metadata": {
|
||||
"id": "YRYFmKxR05xG"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"FRAME_NUMBER = 100\n",
|
||||
"\n",
|
||||
"detections = detections_list[FRAME_NUMBER]\n",
|
||||
"frame_number = detections.data[\"frame_number\"][0]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH, start=FRAME_NUMBER)\n",
|
||||
"frame = next(generator)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "oIK2K9zM1Kn9"
|
||||
},
|
||||
"source": [
|
||||
"###\u00a0Annotate Image with Detections\n",
|
||||
"\n",
|
||||
"Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) classes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 41,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "WBv8apPz1IiP",
|
||||
"outputId": "ad32b078-12d5-4457-e16d-c7a428ab858a"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9+ZMdyXUfjn6yqu7at/cFQHdj34EBZ+MsFEVSlERKFL/62pRk+6uIJ4fseBEO23+Hv/HCv7wIR7wX4Rdhh/yVLGsL2xRJieRoZkjOwuHsGGAGwGBpAI1Gb+j19t2rKt8PWSfrZFbW7QZlSeR7fYbgvV23KiuXk2c/J4WUUmIf9mEf9mEf9mEf9mEf9mEf9mEf9uHnFLx/6A7swz7swz7swz7swz7swz7swz7swz78bWBfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+rmFfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+riHY643/7t/9O0gpM9ellOj1eoiiCADg+77+zfd9eJ4HKSU8T+nQQgj9z/M8xHGMOI71Nd/39W8AEMexfo8QwugDfY/jWL/D931EUWTcT7/VajUUi0Wj//R+aoveS9/t/trP8X7w8XGwnw/DEJ1OB1EUodvt6ja63a6ev16vh2KxiOHhYfi+r8dD7fH30hjDMESr1UKz2cTOzo5+l5QSlUoFxWIRQghEUaTbbLfb2NjYwOzsLL75zW9idHQUL774IsrlMgCgUC5gbHwMvufr90ZRBE96+nscx4iiCFEUQQiBIAjg+74eC++3/d2eL3uM9lg5SCmd74jjGL1eT8+5C28I1+r1OlZXV3HgwAFUq9VMv/hz6+vrWF1dxalTpzJ45FoPwrsgCDL4S/3t9XoaFzzPQ6FQ0GvE59T1HgB6nqltwjEppbEPXO/2fR8ff/wxbt68iV7cw5EjR/Dc888BMSCkhfcCiEWs583zPPS6Pbz1k7dw6+YtlCtlBEGAleUVXL9+HcIT6LQ7OHnyJP7V//1foVQq6T7TOKmdvD3D1zOKIpTLZT1WPiYXbvC54nMRBCbJ4/hFuMzxhj6JRrj6uhegvna7XRSLRQRBgDAMAQCbPziF5tXpx27z7wqCQ+sY/I13IQQMvKG5JNrHaX0/sPe853lot9soFAooFAoG/d0NOM0Pw9C5D/l9dh84EN8iWsef5XhFY45kBBlLQJr7l3gcAecpYRjiypUruHHjBoIgwKlTpzA+Po4oivDmm2/i9u3bABSd/uIXv4jnnntO8xUAGt+bzSauXbuGH//4x1hfX0cYhhpf+RiFEBCBwK/8yq9AxAK/8Au/gJmZGeM+414L/yuVisZzTTuFhIR7fYQQQIInvW4PURyhEBQy7zEfMq97IqEBnkDYCyE8kbuu+p2AWgcpICDwcPkh3n77bSw8XEAcxSiVShgYGMCF8xdw4dwFhGEI3/dRKpXwZ3/2Z/jTP/1TY63z9rTrehRF+Kf/9J/iG9/4BqSUuH37Nv7oj/4I7XZb7+tisYhCoYDz58/jn/2zfwav6EHG6bsEBDx4ml4vLy/jj/7oj/DgwQNIKfXeqFarGtejKML4+DharRY++9nP4h//43+MXq+n+yWFhJDZ/hI+BkHQn2YKAB7g+UqOEhDohT1cv3Ydr/7gVVQqFSwtLqHRbGB1dRWLi4vwpIdu2EWlXMH/+f/4PzE7MwsRC4NO23PJebJNX/Qct33U352BjM3xqH1FcprUeEADUHjqkhU4f44RxymOq77RBBhPGX9Vn3gIf6il+93r9SCl1LiaJ2dkppnRQ6IRrvXgPIevmSETS0/NhxAQAbTspeUP4UHAgxTpM/S853kQUG1HcaR5JN8TElK5v3JYnpQSIkp/jKJI8zWSFQivnBBnplm3K6VELGJFXyEheCdiGH/3kwO4HES4BwBhFEKKHL4j03+utj34iBHD80z8tuVW6hsAJ63eje9xnuuSL+2xt1ot/Jf/8l9w48YN+L6PwcFB/It/8S9w4sQJvP322/iv//W/KjrhCXzjG9/AM888o+YHClc8z4Pnp7qa3RcAxnpzeUDjiw80Gg380R/+kdZvzp8/j6989StoN9t47Yev4cqVKxgfHwcA/Mqv/AqOHj2q8YbWiOuP9O/f/tt/23e+CPas2Lo2KAmcAAyF1BayXYI1PRuGoR4EIWCpVEK5XM4I6nY/uPLAgRRmu+9hGGrFwSYuNvK5kI4TKt4vrpjb93GIogiNRgOtVgvFYhGe56FcLutnBwcHjefu3r0L3/cxMjKS6YfrO1cw6XoYhohjxehJ6eQCUafTQaPRgBACJ0+exDvvvIOTJ0/iwIEDCIIAnW4H7VYbpXIJgR+k42TCnb3RyMCQuzEcirlL+XUB/UYbXW8o1gZXTuhf3rxtb2/ruXfNJ0G324WUEgcPHtTCny0s2n0kvHaNjc+XrUhxZS9PMad/3IDAFX3qG82Ti+Hqd/geSoUSarWa2puIIZDtq/QUc5GxYoSe7+HJzzyJe/fuoV6vo1QsQUIiljE6jQ4838O5c+f0XgZS3CCiRWvDDVk2DtgKJWe8XDixwZ5jF9Pha2a348JvG9f2CqQ40PrwPZqjM/xMw14ZM5Clh7TGhAN5+8jVjv09rw82v3H99tMYKCBgrJcLpziulUolXLp0CadOnTIU3l6vh4MHD+L+/fsIwxAjIyOYnp7WijIJpwQDAwM4f/48FhYWUK/X0el0MvhI39utNnYaO5iemsbg4KAh6BLvtfvfd8iJ8trvd9o/e11LZ9sSEN7jr0mhUMDKygo2tzbhez56rR566KERNbC2ugacS++t1+u4evVqhufsFUgR2dra0rLP6dOn8Vu/9Vv4H//jf+h1oTWen5/H6uoqpg9PI4zDTFvtdhtzc3N4++23MTc3h3PnzqFer2uFlZQebvjc2NiA53mGwK4GkhX2OS5SG/x3Pn7P8+AVPAR+AM9XSo/nefCDhKdEMcrlMjzPw/LSMsjoIYRAu9PG/fv3MTsza7RNgqnNtznPIjCU3E6A+oeHIMO9Gc/+7kGidOwR/KGWHksQBOj1egjDUBtM90oTAXOOwjA0DGR5tMu+RusdyxiITEM/GVDU/W75Sv+eGJtIXvxpgWRMwll6R55hbLe2AIUzGR4voZRdaw/bcqhtGKC29rpGxis5rnpCGRNyaAh/r/0s37e2nGyvO9Frl8xMTg++z+/fv4+7d+/q+Xr++ecxMTGBnZ0dXLt2TT87PT2NkydPQngCQgrEkTLcqpf2wWOm7FNbXHci+a/b7WpcKpfLmD08i263i3a7jTt37mheeOzYMUxNTRlGFRq3S7fbK+xZseUv4wMiRLEVCL6ByBMphGmJ9X0fhUJBt+37PpaXlxFFEarVqvEeThxt5OVMnTNuu6+9Xk9bQglc7bqIif1bFEWaqAGppcz1HI1va2sLGxsbOHjwoFaweR/4WJvNJorFouHpygNbyeFKSxwrJCOFt1Qqwfd9zWDK5TLGxsbQ7XY1gt2+fRvVahUTkxPoRT20O23Dq+EJTyM/N2iQoOoHvjmnibXN3ix8zezrnPFxQusiFLytKIoQBAFKpVIuzvBnDx48iPX1dWxtbWmLdhRFKBaLRh+EEBgdHUWhUMgoinpemHWJK6nUP1v4BVKibSu/XOGz+83H1e8d9vjtdwshsLGxARlLlCtljIyOaI9U5nnFSRAjhozUmgZBgIGBAVy8cBHvvf8eAKDZbGKwNogDUwcwPT2NJ598ElGsPF18f7jGZP/GgXul97Jf+VhtWmHPgS3kut7/0ypCfF8TbgL4WwkPP0uQRytdwK/3i8TZC9g8Jk+5zfvb3m+uNjLKqhRamHK1Rd/5/hdCoFwu63Unhu/7PqanpzExMYH19XWMjIxgcHAQnU5HKw9CKKMTRX10u10sLS2h2WwaPJXvIbp2+tRpPPWZpzAwMGAIRgAy9MJeF1v54UauPBBCeVxJsPHE3gUSrowJoYSsfvtN81SY0TXtVlsbj/22r3ks8bs4jrGwsIC5uTmthPfDuzy8JAGy2Wxqj+qlS5dw48YNfPTRR4YBo9ls4u69u5iankIsYwR+oHBfKD76zjvv4Hvf+x52dnZQrVZx/PhxDA8PY2lpCTdu3NCKLeER8ezJyUl0u11jLbnAacsWXC5yjcv3fRRLRQwMDqBaUbJXq91CHMeoDdQACew0dhCGIdqdNvxA8XpIwA98yFjik48/wckTJzE6PKqeYfPFP20c+2no6j8ESGnOJymkpJS6+Ju7nf6GOH7Ndb+hzEhlaZMQkDI2DGMAtMc2hlsWQvI8jYfolBH1uItSyr2p3EBN/8gDmAd5ih19J0eM5p0yVkYwy9pmy2Wcx3JDkL5PoO/YbLrE+yel8nr24z+2/mEogOx+2/nhklVdijPXv6RU0WA/+clPtFFsZmYGL774IiqVCu7cuYO5uTn9/BMXn8D4+DhiGWfajiPzmvG7hJIRHWOWUjk8QqkiR7u9LsJeiMGhQe2c63a72rkXhiEmJibg+UnkbrKupLBzw4TNc3eDPSu2HGE5gvDNkGGIQmB7exvdbhdDQ0OawbiECa7MjYyMZLxstgDEF5yEeztckPpJ16MoQrvdBgDtIXZBP8GMIxQPyTOIjSXc0DNBECAIArRaLR0qCyATIkT3zs7OIggCpxDsUgSFECiVSigWi5BSYnt7Wyu2cRyj3W5rIYvCaHzfR7VahZQSQ0NDmJ2dxf379zE9PY1KpYLt5jYOHTqEUrmEOEqEUZjvpjEGQaAEMLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"bounding_box_annotator = sv.BoundingBoxAnnotator()\n",
|
||||
"label_annotator = sv.LabelAnnotator()\n",
|
||||
"\n",
|
||||
"annotated_frame = frame.copy()\n",
|
||||
"annotated_frame = bounding_box_annotator.annotate(scene=annotated_frame, detections=detections)\n",
|
||||
"annotated_frame = label_annotator.annotate(scene=annotated_frame, detections=detections)\n",
|
||||
"sv.plot_image(annotated_frame, (12, 12))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "YRPW1JHK1VW-"
|
||||
},
|
||||
"source": [
|
||||
"##\u00a0References \ud83d\udcda\n",
|
||||
"\n",
|
||||
"* Supervision: https://supervision.roboflow.com\n",
|
||||
"* sv.Detections: https://supervision.roboflow.com/develop/detection/core/#detections\n",
|
||||
"* Save Detections to CSV: https://supervision.roboflow.com/develop/how_to/save_detections/#save-detections-as-csv\n",
|
||||
"* Custom fields: https://supervision.roboflow.com/develop/how_to/save_detections/#custom-fields\n",
|
||||
"* ByteTrack: https://supervision.roboflow.com/trackers/#supervision.tracker.byte_tracker.core.ByteTrack\n",
|
||||
"* Inference: https://inference.roboflow.com/\n",
|
||||
"* Inference Pipeline: https://inference.roboflow.com/using_inference/inference_pipeline/\n",
|
||||
"* Inference Aliases: https://inference.roboflow.com/quickstart/aliases/\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "T4",
|
||||
"machine_shape": "hm",
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.14"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
+779
@@ -0,0 +1,779 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RFjrV07Spmm2"
|
||||
},
|
||||
"source": [
|
||||
"# Serialise Detections to a JSON File\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"[](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/detections-to-jsonsink.ipynb)\n",
|
||||
"\n",
|
||||
"This cookbook introduce [sv.JSONSink](https://supervision.roboflow.com/develop/detection/tools/save_detections/#supervision.detection.tools.json_sink.JSONSink) tool designed to write captured object detection data to file from video streams/file"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "aFQHoaDLp8R3"
|
||||
},
|
||||
"source": [
|
||||
"Click the `Open in Colab` button to run the cookbook on Google Colab."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "hzMGlRKlel4p"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install -q inference requests tqdm supervision==0.21.0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"id": "6vRfXc_Je5Ee"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"from typing import List\n",
|
||||
"from collections import defaultdict\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"import supervision as sv\n",
|
||||
"from supervision.assets import download_assets, VideoAssets\n",
|
||||
"from inference import InferencePipeline\n",
|
||||
"from inference.core.interfaces.camera.entities import VideoFrame\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "95av545ydC1O"
|
||||
},
|
||||
"source": [
|
||||
"The parameters defined below are:\n",
|
||||
"* `SOURCE_VIDEO_PATH` - the path to the input video\n",
|
||||
"* `CONFIDENCE_THRESHOLD` - do not include detections below this confidence level\n",
|
||||
"* `IOU_THRESHOLD` - discard detections that overlap with others by more than this [IOU](https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy/) ratio\n",
|
||||
"* `FILE_NAME` - write the json output to this file\n",
|
||||
"* `INFERENCE_MODEL` - model id. This cookbook uses a [model alias](https://inference.roboflow.com/quickstart/aliases/), but it can also be a [fine-tuned model](https://inference.roboflow.com/quickstart/explore_models/) or a model from the [Universe](https://inference.roboflow.com/quickstart/load_from_universe/)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"id": "urRjZjh2f30v"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"people-walking.mp4 asset download complete. \n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"SOURCE_VIDEO_PATH = download_assets(VideoAssets.PEOPLE_WALKING)\n",
|
||||
"CONFIDENCE_THRESHOLD = 0.3\n",
|
||||
"IOU_THRESHOLD = 0.7\n",
|
||||
"FILE_NAME = \"detections.json\"\n",
|
||||
"INFERENCE_MODEL = \"yolov8n-640\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0JjJhzuDAgdH"
|
||||
},
|
||||
"source": [
|
||||
"As a result of executing the above `download_assets(VideoAssets.PEOPLE_WALKING)` , you will download a video file and save it at the `SOURCE_VIDEO_PATH`. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
|
||||
"\n",
|
||||
"<video controls width=\"1280\">\n",
|
||||
" <source src=\"https://media.roboflow.com/supervision/video-examples/people-walking.mp4\" type=\"video/mp4\">\n",
|
||||
"</video>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "HZ20JXd4gYdw"
|
||||
},
|
||||
"source": [
|
||||
"##\u00a0Read single frame from video\n",
|
||||
"\n",
|
||||
"The [`get_video_frames_generator`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.get_video_frames_generator) enables us to easily iterate over video frames. Let's create a video generator for our sample input file and display its first frame on the screen.\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "kRcm6D9agmor",
|
||||
"outputId": "20e3bb60-236c-446c-b636-ce7ea3f591d0"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9aYxm13UeCj/7DO9U89DzzG52N4emJJKiSM2D6UGwEg8IktzERoxc4PvhAAECBMiPAPkd4Lt/vuT+8UWundgZbMWyJVmWYFoWZYqSSIrsJrubYpPdzZ6qu6qrqmt+x3PO/n7ss9ZZe5993qpm7ocPBmo3GlX1vufsYe2117zXUlprjd2223bbbtttu2237bbdttt2227bbbvt72gL/v89gd2223bbbtttu2237bbdttt2227bbbvtf6XtKra7bbfttt2223bbbtttu2237bbdttv+TrddxXa37bbdttt2227bbbttt+223bbbdtvf6bar2O623bbbdttu2227bbfttt2223bbbvs73XYV292223bbbtttu2237bbdttt2227bbX+n265iu9t2227bbbttt+223bbbdttu22277e9021Vsd9tu2227bbfttt2223bbbtttu223/Z1uu4rtbtttu2237bbdttt2227bbbttt+22v9NtV7Hdbbttt+223bbbdttu2227bbfttt32d7pFO33w3/27fzf0e6UU/x/WtNbQWlvPK6UQBAGiKEIQFLp2VZ/UBwBkWYYkSZCmaalf6gMAgiCw+nHHj6IIURSVnvHNXynFc8iyjPv4H//jf+DixYt45plncOTIEUxNTaHVavHYNGetNTY3N5EkCTY2NjA3N4eDBw/i4x//OKKo2JIwDC14yLVnWcZ/07Pbzd3th+BHv8t35D5J2NHnNAf3/SzLMBgMkCQJf5dlGS5cuIC//uu/Rr1ex969e/GFL3wBjUYDt27dwttvv4379+/z/lN/cRzjl37pl/Cxj30MaZpa+ypxgNYfhiH++3//73j//fdx8OBBjI+Po9PpIIoi1Ot1JEmCwWCATqeDXq+HlZUVRFGEyclJ/Mt/+S8xNTXFc1ZKMUyrYCn3IY5jhGFozWm7vdBaI01TAEAURQjDELVazfsO4TmNmaYpkiRBkiQ81zAMuV/aM9q/IAh4PXK/lFLch3tGfPOXOCy/j+MYURR51+/7jD6XMIjj2JpfVT/yp9Yag8EAWZZZZ2cn77ut0WjYa4UGFMz/IS3NzF7EUTx0r32fZbqgX/V6ffictb8fel8phWazibv37uIP/8sfoj/oQ2caURwhCg1sRkZH8Pf/3t/H9PR09Tjis7gW40//55/iL//yLxHHMV588UW88MILmJycxH/4j/8B165ew6lTp/Av/sW/QK1es+hhoHZoN1WAzjTCMEQcx0MfdedHLdPm3MZR7P2+qmVZhrm7c1hfW8fS0hJu3b6Fffv2YXRkFFEclfZTZ+b8zczO4JFHHrHGytIMtXrNotk600PnIftP0gQAGA/c99yzoZRCGBiao2HG6ff6aLVaNv9zUJJwqfQZgF6vhyAI+CxCofTssEZ0KYxCKCjeF996fX9nWYZkkBg66DlKVfQ1UAW9S9MUmc4QRzFChJVzHbYvg8GAZQKal4t7kv5Tk7w+TVM0Gg1+3uXlcv5VcyC65pNjquYv++v3+1Yf/F2AEnxLPB4aaZJCQzNtc3mHi5MWbYaGgkK/34eGNjxOhQhyf4qXJkJDK/vzQWLkiSiMENdiLz7SPLIsw1vn38L/9Xv/F7Isw4EDB/Drv/7r+O3f+m2Mj43jvSvv4b/80X/B4UOHMTY2hnq9jg8//BCnz5zGxMQEsjTjfcq0vbdaG3hAgemUi9sKDv/wtCzLTD/0TqDoBS9cpKxD55zeJbzXWiNJE+hMyHJK7BOKvfKti9+pWAfjRE7PgjBgeKRZWnqfYeIgmZy/hAc0oAJDz9w+JH1z5yz/ztIMmTYyaRAEhufl9KuKBrlypEuvZOMzmPO1Kv1A9uOj11Xvuv0kacJ4UjVv2sssy3jvAcNLkoGRF9Ms5e96/R5++PIPcfHiRea7WWbgJt9353H69Gn8w3/4D1k+pzNiyeBBaOi+Ky+KbklnYDjTd4p+5PREAXEUIwgDi9b87v/rd4fCjdqOFdvt2k6Yl1Q85XfbCd6+Pl3FzjqUvgPmKLLyM59SK5HSp1i7PxuNBj73uc/h4sWL+OlPf4rz588jiiKMjo6i2WwiiiL0ej0kSYJ2u41Op4MjR47g6aefxsTEhFeJGob8BCcJP/d7lwG6e1Qiyg7cpHJb1a/7HiF9EAQW0z916hR6vR5GR0cxNTWFIAhw9+5dvP7669ja2kKj0cDU1BTCMMT9+/etcX0EwoVPo9HA8vIy2u02RkZGmPGEYYjBYMD7LPccACYnJ5FlGRYWFjAzM2ONsZ0QIRXH7ZSqKoUxSRKGl1TsfXgtYZGmqRHgssxSaAFYSq0Phj6msB2hrWoSB6vOzDBlj/67a5DPuL+TACOVfGkQ8eH2do0MCpXN7UIIIZL5PGwjYaFK6HWflT+tPnJhI8sy1Go1xDVjaJBMTSmFLM1w89ZNTE5O+o1mQoDIsgxhGqLX7xkGlyS4fec2ntfPI67FOH78OK5fu44rV67g9Tdex5e++CV+VmttBDZlCzdeYTYzzDMIAqjAzNG7fpqbFkKjBqAK4agkNGl/X0DBRLM0Nxrlwne/10c/7hd0VVXgcz52aX5D2k5wcZhxxzd/V4Am2CuUlVr5jKdDQBnlNI5j0w92OBdBZ9IsRaQiM744J1Vrl/gRqMAoVFnKNIG+9ykKUggvnYuH0ciddYRhyAY3mof7HK2pSuAmQ6qkbTsRaH00lGjkdkqt/I7osmsU3s5QJ58N4sCisb5zoAJlCcWW8qRyWGbG8aCVvb7tcEtrjSiMeD/ctShlcGyQDBAGIcsUG5sbqMU1DAYDXLhwAZOTk/jiF76IH//kx3j55ZfRbrehM42nn3kaX/2Vr2JmegaZzixc8s0vCAOzVkGDpOIo4e82VogCBR3qSkXC/bvqjAM5fctprFKKYzCrlFRf8z27Ezyj51zeK2FReh6qtBYVKms99Ix8dqfzyFKjoGXIoIOCfv2vyDjbNVdfIbqZpmmJd9Bz2zWGawWelMaX8kiWWjyU8CLLMsRxjE9/+tO4efMmVldXvQp4aS5Q2L9vP5qNJgbJoFJ+dPUXGtvab2UcAVmW2XiSP5IhN3JsR6S2af+PKbY7aT5C4SoQvgPlElMSalmgG6KASMVPAp/eC8PQUmp9B6CKyFmWnizDmTNn8MILL+Cll14CYASEdruNMAxx4MAB3Llzh61JgLHIHj9+HJ1OBxMTE5a3Yjvkl2vxrd+FWRXBdPuTypFPGXKfc8chBY2eSRLjhajVanjiiSewtbWFwWCAwWCAkZERHD16FFEUYXx8HGNjY1haWsLy8jIzMIKtuy9ScabPr169itXVVWs+tVoN9XodnU6H4UQCQr1ex8zMDAaDAT744AM88cQTvG4J32FKGjF8gpErjPjekd9nWYZGo1FSaocxFhI0SPBycVrumVReJExdGLpzlvOuwn/ZT9Vct2vy/A4TFOVP+k9wkHMY1uQY7j5V7aF81zcn11q93djW2pFbwKHZo7pd851b+kkW6larxUakJEnQ6/agoRGqEEma4NbNW3jsscfQbDTtOeXWbcAom2maYnljGUtLS1CBYdSD/gBplkJBodloMvzff/99vPDCCyX8VWpnnluOQhgktjLK+qvAA1JgxXwZD8LyWFW8RMPQp36/j263izAIMTExgV6vV60UER/WhUdKzlVnRnhnL0w+jisw+/qVz/j4mLsmOQYpsQx/IRz64OA9q9r8tzzOOzzT8lxlqdnLMAwRqACpTq3nfOuRf2dphoE2igoJhrSm7RR1hq82/WRBVhKSdnJWgyBg3iX5rI+2Vu1VGIZseNS6iExxn5P0SMo2gJERAL/hbSdKoc/oDaCMu3JOJIzrQsHfam+hHtYL+OTnLEszqEAxL+H9JaVEG0UuQNnwWSnrKc98UMCH8IG8X5k2ML7x4Q38H//v/wP3Fu4BMB6kjY0N3L5zG3/913+Nd955B81mE888/Qxe/uHLWFtdw6uvvoozZ87g8OHDxruti4gTlw8HKjCKudL8HK1VrsNrgBGLYiUs9y4is/nDw5w5opW0x3JOPkWDnvGN4eONhAfuuizjCQKjSDpn2ze2l/5JAyEZDPKfw+Tu0niBQqQig3NiD13a8FHkFHpf8rLt+iGvtpwjNZ884TYF4+HNVGYU9ornpDyUaRPxwp/n55zOUZIlgAKmZ6axtbWFdqftNV66ODAzM1OSFd25+/DN3W+ScweDgVHYla2YkyL8v2qE+H9csd0OmV3h0iVwLvEDbO9s1QZUEUn63KfY+pTaqjUppbwbS20wGCAIAnzmM5/B+fPnsbGxwZsYhiG+/OUv48KFC7hw4QIzgW63i9HR0ZKnUI5ZSfzzJhmXC2cfTH0KgjuOSySrlGKfgiAFALL6A2CPNYW4BUGARqOBZ555hvdjMBhgbW3NWjutTxonaCwK1W40Grh8+TK+//3vY21tDWNjYzxHEioajQYLzzSner0OpUz45vvvv4+1tTVMT0+zF1Wuz7dWqZS5+zLsbym01Ov1Iuyv4j13z6RC586R9oAEACs01BFwCJ99IcQuvvmEYhp3J2eoqsl5SsVenrVhyq5cn2xVhNEH54/K6Ijpk4fL/c5Hqyy4QrGA5F5X8DUShKy15YJjp92BUgr1sI6R1gjGx8cxd2cOMzMzCIIA/UEfYc0YdFZWVnDn9h2cOXOmJIwCZj/X1tbw+uuv4+WXX8bKygr27duHBw8eYHp6GlOTU9jc2sTt27f53bk7c1heXkar1UIYmHOZBRkCZX6W4EbTp3OkgoLJ5esyP/K9zoTnILine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH)\n",
|
||||
"frame = next(generator)\n",
|
||||
"sv.plot_image(frame, (12, 12))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6yeYawkyE4wv"
|
||||
},
|
||||
"source": [
|
||||
"We can also use [`VideoInfo.from_video_path`](https://supervision.roboflow.com/develop/utils/video/#supervision.utils.video.VideoInfo) to learn basic information about our video, such as duration, resolution, or FPS."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "pVN_fhdGE3hD",
|
||||
"outputId": "a05c7c1c-8bcd-46f7-d404-7b766b2cc0e5"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"VideoInfo(width=1920, height=1080, fps=25, total_frames=341)"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"sv.VideoInfo.from_video_path(SOURCE_VIDEO_PATH)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "2YyE30JOS4Sf"
|
||||
},
|
||||
"source": [
|
||||
"## Initialize ByteTrack\n",
|
||||
"\n",
|
||||
"[ByteTrack](https://supervision.roboflow.com/latest/trackers/#bytetrack) is a multi-object tracking algorithm used by Supervision to track and link detected objects across multiple frames, providing consistent IDs for each object."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"id": "Jpw_TzAsm2oL"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"byte_track = sv.ByteTrack(minimum_consecutive_frames=3)\n",
|
||||
"byte_track.reset()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "IjYwj2vbTLBd"
|
||||
},
|
||||
"source": [
|
||||
"## Initialize sv.JSONSink\n",
|
||||
"\n",
|
||||
"To save detections to a `JSON` file, open our [`sv.JSONSink`](https://supervision.roboflow.com/latest/how_to/save_detections/#save-detections-as-json) and then pass the [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) object resulting from the inference to it.\n",
|
||||
"\n",
|
||||
"Note that empty detections will be skipped."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 40,
|
||||
"metadata": {
|
||||
"id": "-yvv6N1uTKSl"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"json_sink = sv.JSONSink(FILE_NAME)\n",
|
||||
"json_sink.open()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "-YvmJW5yud1G"
|
||||
},
|
||||
"source": [
|
||||
"## Process video and save detections to json file\n",
|
||||
"\n",
|
||||
"The [`InferencePipeline`](https://inference.roboflow.com/using_inference/inference_pipeline/) interface is made for streaming and is likely the best route to go for real time use cases. It is an asynchronous interface that can consume many different video sources including local devices (like webcams), RTSP video streams, video files, etc. With this interface, you define the source of a video stream and sinks.\n",
|
||||
"\n",
|
||||
"All the operations we plan to perform for each frame of our video - detection, tracking, annotation, and write to json - are encapsulated in a function named `callback`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 41,
|
||||
"metadata": {
|
||||
"id": "eRQiVkblapCk"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def callback(predictions: dict, frame: VideoFrame) -> np.ndarray:\n",
|
||||
" detections = sv.Detections.from_inference(predictions)\n",
|
||||
" \n",
|
||||
" # Only keep person detections\n",
|
||||
" detections = detections[detections.class_id == 0]\n",
|
||||
" detections.data[\"class_name\"] = np.array([\"person\" for _ in range(len(detections))])\n",
|
||||
"\n",
|
||||
" detections = byte_track.update_with_detections(detections)\n",
|
||||
" json_sink.append(detections, custom_data={'frame_number': frame.frame_id})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "3GRH_QdHgGiC",
|
||||
"outputId": "69ec2da5-73d3-4a74-c9de-ebd8080f5bc7"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pipeline = InferencePipeline.init(\n",
|
||||
" model_id=INFERENCE_MODEL,\n",
|
||||
" video_reference=SOURCE_VIDEO_PATH,\n",
|
||||
" on_prediction=callback,\n",
|
||||
" iou_threshold=IOU_THRESHOLD,\n",
|
||||
" confidence=CONFIDENCE_THRESHOLD,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"metadata": {
|
||||
"id": "bYpA9VxBsOvS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pipeline.start()\n",
|
||||
"pipeline.join()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 44,
|
||||
"metadata": {
|
||||
"id": "KXbZt-FNVnrS"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"json_sink.write_and_close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "u7Akx7aUsh75"
|
||||
},
|
||||
"source": [
|
||||
"## Visualizate results of detections json data with Pandas\n",
|
||||
"\n",
|
||||
"Let's take a look at our resulting data with by using Pandas.\n",
|
||||
"\n",
|
||||
"It will also be created in your current directory with the name detections.json as well."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 45,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 424
|
||||
},
|
||||
"id": "Bu7AZ3QHiAqb",
|
||||
"outputId": "4455e1bd-f42a-401b-9358-36d4964ca7eb"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<div>\n",
|
||||
"<style scoped>\n",
|
||||
" .dataframe tbody tr th:only-of-type {\n",
|
||||
" vertical-align: middle;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe tbody tr th {\n",
|
||||
" vertical-align: top;\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" .dataframe thead th {\n",
|
||||
" text-align: right;\n",
|
||||
" }\n",
|
||||
"</style>\n",
|
||||
"<table border=\"1\" class=\"dataframe\">\n",
|
||||
" <thead>\n",
|
||||
" <tr style=\"text-align: right;\">\n",
|
||||
" <th></th>\n",
|
||||
" <th>x_min</th>\n",
|
||||
" <th>y_min</th>\n",
|
||||
" <th>x_max</th>\n",
|
||||
" <th>y_max</th>\n",
|
||||
" <th>class_id</th>\n",
|
||||
" <th>confidence</th>\n",
|
||||
" <th>tracker_id</th>\n",
|
||||
" <th>class_name</th>\n",
|
||||
" <th>frame_number</th>\n",
|
||||
" </tr>\n",
|
||||
" </thead>\n",
|
||||
" <tbody>\n",
|
||||
" <tr>\n",
|
||||
" <th>0</th>\n",
|
||||
" <td>1142</td>\n",
|
||||
" <td>950</td>\n",
|
||||
" <td>1245</td>\n",
|
||||
" <td>1080</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.767850</td>\n",
|
||||
" <td>365</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>1</th>\n",
|
||||
" <td>750</td>\n",
|
||||
" <td>450</td>\n",
|
||||
" <td>823</td>\n",
|
||||
" <td>620</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.748268</td>\n",
|
||||
" <td>348</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>2</th>\n",
|
||||
" <td>1419</td>\n",
|
||||
" <td>702</td>\n",
|
||||
" <td>1526</td>\n",
|
||||
" <td>887</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.550286</td>\n",
|
||||
" <td>359</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>3</th>\n",
|
||||
" <td>1674</td>\n",
|
||||
" <td>12</td>\n",
|
||||
" <td>1727</td>\n",
|
||||
" <td>141</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.546864</td>\n",
|
||||
" <td>354</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>4</th>\n",
|
||||
" <td>1616</td>\n",
|
||||
" <td>18</td>\n",
|
||||
" <td>1655</td>\n",
|
||||
" <td>127</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.520566</td>\n",
|
||||
" <td>361</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>1</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>...</th>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" <td>...</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8757</th>\n",
|
||||
" <td>394</td>\n",
|
||||
" <td>471</td>\n",
|
||||
" <td>460</td>\n",
|
||||
" <td>633</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.466400</td>\n",
|
||||
" <td>424</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8758</th>\n",
|
||||
" <td>115</td>\n",
|
||||
" <td>256</td>\n",
|
||||
" <td>199</td>\n",
|
||||
" <td>399</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.431374</td>\n",
|
||||
" <td>403</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8759</th>\n",
|
||||
" <td>77</td>\n",
|
||||
" <td>700</td>\n",
|
||||
" <td>169</td>\n",
|
||||
" <td>892</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.416318</td>\n",
|
||||
" <td>420</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8760</th>\n",
|
||||
" <td>1010</td>\n",
|
||||
" <td>59</td>\n",
|
||||
" <td>1055</td>\n",
|
||||
" <td>174</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.403422</td>\n",
|
||||
" <td>426</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" <tr>\n",
|
||||
" <th>8761</th>\n",
|
||||
" <td>1758</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>1809</td>\n",
|
||||
" <td>96</td>\n",
|
||||
" <td>0</td>\n",
|
||||
" <td>0.313793</td>\n",
|
||||
" <td>416</td>\n",
|
||||
" <td>person</td>\n",
|
||||
" <td>341</td>\n",
|
||||
" </tr>\n",
|
||||
" </tbody>\n",
|
||||
"</table>\n",
|
||||
"<p>8762 rows \u00d7 9 columns</p>\n",
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" x_min y_min x_max y_max class_id confidence tracker_id class_name \\\n",
|
||||
"0 1142 950 1245 1080 0 0.767850 365 person \n",
|
||||
"1 750 450 823 620 0 0.748268 348 person \n",
|
||||
"2 1419 702 1526 887 0 0.550286 359 person \n",
|
||||
"3 1674 12 1727 141 0 0.546864 354 person \n",
|
||||
"4 1616 18 1655 127 0 0.520566 361 person \n",
|
||||
"... ... ... ... ... ... ... ... ... \n",
|
||||
"8757 394 471 460 633 0 0.466400 424 person \n",
|
||||
"8758 115 256 199 399 0 0.431374 403 person \n",
|
||||
"8759 77 700 169 892 0 0.416318 420 person \n",
|
||||
"8760 1010 59 1055 174 0 0.403422 426 person \n",
|
||||
"8761 1758 0 1809 96 0 0.313793 416 person \n",
|
||||
"\n",
|
||||
" frame_number \n",
|
||||
"0 1 \n",
|
||||
"1 1 \n",
|
||||
"2 1 \n",
|
||||
"3 1 \n",
|
||||
"4 1 \n",
|
||||
"... ... \n",
|
||||
"8757 341 \n",
|
||||
"8758 341 \n",
|
||||
"8759 341 \n",
|
||||
"8760 341 \n",
|
||||
"8761 341 \n",
|
||||
"\n",
|
||||
"[8762 rows x 9 columns]"
|
||||
]
|
||||
},
|
||||
"execution_count": 45,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"df = pd.read_json(FILE_NAME)\n",
|
||||
"df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TKxN1TDE8ssv"
|
||||
},
|
||||
"source": [
|
||||
"## Convert JSON data to sv.Detections\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 46,
|
||||
"metadata": {
|
||||
"id": "iwJf7PW6eLkU"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def json_to_detections(json_file: str) -> List[sv.Detections]:\n",
|
||||
" rows_by_frame_number = defaultdict(list)\n",
|
||||
" with open(json_file, \"r\") as f:\n",
|
||||
" data = json.load(f)\n",
|
||||
" for row in data:\n",
|
||||
" frame_number = int(row[\"frame_number\"])\n",
|
||||
" rows_by_frame_number[frame_number].append(row)\n",
|
||||
"\n",
|
||||
" detections_list = []\n",
|
||||
" for frame_number, rows in rows_by_frame_number.items():\n",
|
||||
" xyxy = []\n",
|
||||
" class_id = []\n",
|
||||
" confidence = []\n",
|
||||
" tracker_id = []\n",
|
||||
" custom_data = defaultdict(list)\n",
|
||||
"\n",
|
||||
" for row in rows:\n",
|
||||
" xyxy.append([row[key] for key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\"]])\n",
|
||||
" class_id.append(row[\"class_id\"])\n",
|
||||
" confidence.append(row[\"confidence\"])\n",
|
||||
" tracker_id.append(row[\"tracker_id\"])\n",
|
||||
"\n",
|
||||
" for custom_key in row.keys():\n",
|
||||
" if custom_key in [\"x_min\", \"y_min\", \"x_max\", \"y_max\", \"class_id\", \"confidence\", \"tracker_id\"]:\n",
|
||||
" continue\n",
|
||||
" custom_data[custom_key].append(row[custom_key])\n",
|
||||
"\n",
|
||||
" if all([val == \"\" for val in class_id]):\n",
|
||||
" class_id = None\n",
|
||||
" if all([val == \"\" for val in confidence]):\n",
|
||||
" confidence = None\n",
|
||||
" if all([val == \"\" for val in tracker_id]):\n",
|
||||
" tracker_id = None\n",
|
||||
"\n",
|
||||
" detections_list.append(\n",
|
||||
" sv.Detections(\n",
|
||||
" xyxy=np.array(xyxy, dtype=np.float32),\n",
|
||||
" class_id=np.array(class_id, dtype=int),\n",
|
||||
" confidence=np.array(confidence, dtype=np.float32),\n",
|
||||
" tracker_id=np.array(tracker_id, dtype=int),\n",
|
||||
" data=dict(custom_data)\n",
|
||||
" )\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" return detections_list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 47,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "TRU2_c3tlBHT",
|
||||
"outputId": "17801131-fdc5-4c5d-f753-af9c557a4539"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Detections: 341\n",
|
||||
"Detections(xyxy=array([[1142., 950., 1245., 1080.],\n",
|
||||
" [ 750., 450., 823., 620.],\n",
|
||||
" [1419., 702., 1526., 887.],\n",
|
||||
" [1674., 12., 1727., 141.],\n",
|
||||
" [1616., 18., 1655., 127.],\n",
|
||||
" [1388., 2., 1437., 184.]], dtype=float32), mask=None, confidence=array([0.7678498 , 0.7482683 , 0.5502863 , 0.5468636 , 0.5205659 ,\n",
|
||||
" 0.31684005], dtype=float32), class_id=array([0, 0, 0, 0, 0, 0]), tracker_id=array([365, 348, 359, 354, 361, 28]), data={'class_name': ['person', 'person', 'person', 'person', 'person', 'person'], 'frame_number': [1, 1, 1, 1, 1, 1]})\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"detections_list = json_to_detections(FILE_NAME)\n",
|
||||
"detections_list\n",
|
||||
"\n",
|
||||
"print(f\"Detections: {len(detections_list)}\")\n",
|
||||
"print(detections_list[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "ZlvzmExpqdvI"
|
||||
},
|
||||
"source": [
|
||||
"###\u00a0Annotate First Frame\n",
|
||||
"\n",
|
||||
"Visualize the first frame of a video alongside the initial detections obtained by parsing JSON data into [`sv.Detections`](https://supervision.roboflow.com/latest/detection/core/#detections) objects. The annotated image will show the original video frame, marked with the first bounding box detected from the parsed data, providing a visual representation of the identified object(s) in the scene."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "SqER9kISrW_0"
|
||||
},
|
||||
"source": [
|
||||
"### Get back `sv.Detections`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 48,
|
||||
"metadata": {
|
||||
"id": "I18IZQuFrPeN"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"FRAME_NUMBER = 100\n",
|
||||
"\n",
|
||||
"detections = detections_list[FRAME_NUMBER]\n",
|
||||
"frame_number = detections.data[\"frame_number\"][0]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"generator = sv.get_video_frames_generator(SOURCE_VIDEO_PATH, start=FRAME_NUMBER)\n",
|
||||
"frame = next(generator)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yTOQxeeCraYm"
|
||||
},
|
||||
"source": [
|
||||
"### First frame from video (Before Annotate)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "FK5VfyJurhEY"
|
||||
},
|
||||
"source": [
|
||||
"###\u00a0Annotate Image with Detections\n",
|
||||
"\n",
|
||||
"Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](https://supervision.roboflow.com/latest/detection/annotators/#supervision.annotators.core.LabelAnnotator) classes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 49,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 560
|
||||
},
|
||||
"id": "WuXAYqLBqBH8",
|
||||
"outputId": "1bd7003c-62dc-4860-bc94-86e0995f92a0"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAIfCAYAAABEnkAKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9+ZMdyXUfjn6yqu7at/cFQHdj34EBZ+MsFEVSlERKFL/62pRk+6uIJ4fseBEO23+Hv/HCv7wIR7wX4Rdhh/yVLGsL2xRJieRoZkjOwuHsGGAGwGBpAI1Gb+j19t2rKt8PWSfrZFbW7QZlSeR7fYbgvV23KiuXk2c/J4WUUmIf9mEf9mEf9mEf9mEf9mEf9mEf9uHnFLx/6A7swz7swz7swz7swz7swz7swz7swz78bWBfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+rmFfsd2HfdiHfdiHfdiHfdiHfdiHfdiHn2vYV2z3YR/2YR/2YR/2YR/2YR/2YR/24eca9hXbfdiHfdiHfdiHfdiHfdiHfdiHffi5hn3Fdh/2YR/2YR/2YR/2YR/2YR/2YR9+riHY643/7t/9O0gpM9ellOj1eoiiCADg+77+zfd9eJ4HKSU8T+nQQgj9z/M8xHGMOI71Nd/39W8AEMexfo8QwugDfY/jWL/D931EUWTcT7/VajUUi0Wj//R+aoveS9/t/trP8X7w8XGwnw/DEJ1OB1EUodvt6ja63a6ev16vh2KxiOHhYfi+r8dD7fH30hjDMESr1UKz2cTOzo5+l5QSlUoFxWIRQghEUaTbbLfb2NjYwOzsLL75zW9idHQUL774IsrlMgCgUC5gbHwMvufr90ZRBE96+nscx4iiCFEUQQiBIAjg+74eC++3/d2eL3uM9lg5SCmd74jjGL1eT8+5C28I1+r1OlZXV3HgwAFUq9VMv/hz6+vrWF1dxalTpzJ45FoPwrsgCDL4S/3t9XoaFzzPQ6FQ0GvE59T1HgB6nqltwjEppbEPXO/2fR8ff/wxbt68iV7cw5EjR/Dc888BMSCkhfcCiEWs583zPPS6Pbz1k7dw6+YtlCtlBEGAleUVXL9+HcIT6LQ7OHnyJP7V//1foVQq6T7TOKmdvD3D1zOKIpTLZT1WPiYXbvC54nMRBCbJ4/hFuMzxhj6JRrj6uhegvna7XRSLRQRBgDAMAQCbPziF5tXpx27z7wqCQ+sY/I13IQQMvKG5JNrHaX0/sPe853lot9soFAooFAoG/d0NOM0Pw9C5D/l9dh84EN8iWsef5XhFY45kBBlLQJr7l3gcAecpYRjiypUruHHjBoIgwKlTpzA+Po4oivDmm2/i9u3bABSd/uIXv4jnnntO8xUAGt+bzSauXbuGH//4x1hfX0cYhhpf+RiFEBCBwK/8yq9AxAK/8Au/gJmZGeM+414L/yuVisZzTTuFhIR7fYQQQIInvW4PURyhEBQy7zEfMq97IqEBnkDYCyE8kbuu+p2AWgcpICDwcPkh3n77bSw8XEAcxSiVShgYGMCF8xdw4dwFhGEI3/dRKpXwZ3/2Z/jTP/1TY63z9rTrehRF+Kf/9J/iG9/4BqSUuH37Nv7oj/4I7XZb7+tisYhCoYDz58/jn/2zfwav6EHG6bsEBDx4ml4vLy/jj/7oj/DgwQNIKfXeqFarGtejKML4+DharRY++9nP4h//43+MXq+n+yWFhJDZ/hI+BkHQn2YKAB7g+UqOEhDohT1cv3Ydr/7gVVQqFSwtLqHRbGB1dRWLi4vwpIdu2EWlXMH/+f/4PzE7MwsRC4NO23PJebJNX/Qct33U352BjM3xqH1FcprUeEADUHjqkhU4f44RxymOq77RBBhPGX9Vn3gIf6il+93r9SCl1LiaJ2dkppnRQ6IRrvXgPIevmSETS0/NhxAQAbTspeUP4UHAgxTpM/S853kQUG1HcaR5JN8TElK5v3JYnpQSIkp/jKJI8zWSFQivnBBnplm3K6VELGJFXyEheCdiGH/3kwO4HES4BwBhFEKKHL4j03+utj34iBHD80z8tuVW6hsAJ63eje9xnuuSL+2xt1ot/Jf/8l9w48YN+L6PwcFB/It/8S9w4sQJvP322/iv//W/KjrhCXzjG9/AM888o+YHClc8z4Pnp7qa3RcAxnpzeUDjiw80Gg380R/+kdZvzp8/j6989StoN9t47Yev4cqVKxgfHwcA/Mqv/AqOHj2q8YbWiOuP9O/f/tt/23e+CPas2Lo2KAmcAAyF1BayXYI1PRuGoR4EIWCpVEK5XM4I6nY/uPLAgRRmu+9hGGrFwSYuNvK5kI4TKt4vrpjb93GIogiNRgOtVgvFYhGe56FcLutnBwcHjefu3r0L3/cxMjKS6YfrO1cw6XoYhohjxehJ6eQCUafTQaPRgBACJ0+exDvvvIOTJ0/iwIEDCIIAnW4H7VYbpXIJgR+k42TCnb3RyMCQuzEcirlL+XUB/UYbXW8o1gZXTuhf3rxtb2/ruXfNJ0G324WUEgcPHtTCny0s2n0kvHaNjc+XrUhxZS9PMad/3IDAFX3qG82Ti+Hqd/geSoUSarWa2puIIZDtq/QUc5GxYoSe7+HJzzyJe/fuoV6vo1QsQUIiljE6jQ4838O5c+f0XgZS3CCiRWvDDVk2DtgKJWe8XDixwZ5jF9Pha2a348JvG9f2CqQ40PrwPZqjM/xMw14ZM5Clh7TGhAN5+8jVjv09rw82v3H99tMYKCBgrJcLpziulUolXLp0CadOnTIU3l6vh4MHD+L+/fsIwxAjIyOYnp7WijIJpwQDAwM4f/48FhYWUK/X0el0MvhI39utNnYaO5iemsbg4KAh6BLvtfvfd8iJ8trvd9o/e11LZ9sSEN7jr0mhUMDKygo2tzbhez56rR566KERNbC2ugacS++t1+u4evVqhufsFUgR2dra0rLP6dOn8Vu/9Vv4H//jf+h1oTWen5/H6uoqpg9PI4zDTFvtdhtzc3N4++23MTc3h3PnzqFer2uFlZQebvjc2NiA53mGwK4GkhX2OS5SG/x3Pn7P8+AVPAR+AM9XSo/nefCDhKdEMcrlMjzPw/LSMsjoIYRAu9PG/fv3MTsza7RNgqnNtznPIjCU3E6A+oeHIMO9Gc/+7kGidOwR/KGWHksQBOj1egjDUBtM90oTAXOOwjA0DGR5tMu+RusdyxiITEM/GVDU/W75Sv+eGJtIXvxpgWRMwll6R55hbLe2AIUzGR4voZRdaw/bcqhtGKC29rpGxis5rnpCGRNyaAh/r/0s37e2nGyvO9Frl8xMTg++z+/fv4+7d+/q+Xr++ecxMTGBnZ0dXLt2TT87PT2NkydPQngCQgrEkTLcqpf2wWOm7FNbXHci+a/b7WpcKpfLmD08i263i3a7jTt37mheeOzYMUxNTRlGFRq3S7fbK+xZseUv4wMiRLEVCL6ByBMphGmJ9X0fhUJBt+37PpaXlxFFEarVqvEeThxt5OVMnTNuu6+9Xk9bQglc7bqIif1bFEWaqAGppcz1HI1va2sLGxsbOHjwoFaweR/4WJvNJorFouHpygNbyeFKSxwrJCOFt1Qqwfd9zWDK5TLGxsbQ7XY1gt2+fRvVahUTkxPoRT20O23Dq+EJTyM/N2iQoOoHvjmnibXN3ix8zezrnPFxQusiFLytKIoQBAFKpVIuzvBnDx48iPX1dWxtbWmLdhRFKBaLRh+EEBgdHUWhUMgoinpemHWJK6nUP1v4BVKibSu/XOGz+83H1e8d9vjtdwshsLGxARlLlCtljIyOaI9U5nnFSRAjhozUmgZBgIGBAVy8cBHvvf8eAKDZbGKwNogDUwcwPT2NJ598ElGsPF18f7jGZP/GgXul97Jf+VhtWmHPgS3kut7/0ypCfF8TbgL4WwkPP0uQRytdwK/3i8TZC9g8Jk+5zfvb3m+uNjLKqhRamHK1Rd/5/hdCoFwu63Unhu/7PqanpzExMYH19XWMjIxgcHAQnU5HKw9CKKMTRX10u10sLS2h2WwaPJXvIbp2+tRpPPWZpzAwMGAIRgAy9MJeF1v54UauPBBCeVxJsPHE3gUSrowJoYSsfvtN81SY0TXtVlsbj/22r3ks8bs4jrGwsIC5uTmthPfDuzy8JAGy2Wxqj+qlS5dw48YNfPTRR4YBo9ls4u69u5iankIsYwR+oHBfKD76zjvv4Hvf+x52dnZQrVZx/PhxDA8PY2lpCTdu3NCKLeER8ezJyUl0u11jLbnAacsWXC5yjcv3fRRLRQwMDqBaUbJXq91CHMeoDdQACew0dhCGIdqdNvxA8XpIwA98yFjik48/wckTJzE6PKqeYfPFP20c+2no6j8ESGnOJymkpJS6+Ju7nf6GOH7Ndb+hzEhlaZMQkDI2DGMAtMc2hlsWQvI8jYfolBH1uItSyr2p3EBN/8gDmAd5ih19J0eM5p0yVkYwy9pmy2Wcx3JDkL5PoO/YbLrE+yel8nr24z+2/mEogOx+2/nhklVdijPXv6RU0WA/+clPtFFsZmYGL774IiqVCu7cuYO5uTn9/BMXn8D4+DhiGWfajiPzmvG7hJIRHWOWUjk8QqkiR7u9LsJeiMGhQe2c63a72rkXhiEmJibg+UnkbrKupLBzw4TNc3eDPSu2HGE5gvDNkGGIQmB7exvdbhdDQ0OawbiECa7MjYyMZLxstgDEF5yEeztckPpJ16MoQrvdBgDtIXZBP8GMIxQPyTOIjSXc0DNBECAIArRaLR0qCyATIkT3zs7OIggCpxDsUgSFECiVSigWi5BSYnt7Wyu2cRyj3W5rIYvCaHzfR7VahZQSQ0NDmJ2dxf379zE9PY1KpYLt5jYOHTqEUrmEOEqEUZjvpjEGQaAEMLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1200x1200 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"bounding_box_annotator = sv.BoundingBoxAnnotator()\n",
|
||||
"label_annotator = sv.LabelAnnotator()\n",
|
||||
"\n",
|
||||
"annotated_frame = frame.copy()\n",
|
||||
"annotated_frame = bounding_box_annotator.annotate(scene=annotated_frame, detections=detections)\n",
|
||||
"annotated_frame = label_annotator.annotate(scene=annotated_frame, detections=detections)\n",
|
||||
"sv.plot_image(annotated_frame, (12, 12))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "975K127Regdr"
|
||||
},
|
||||
"source": [
|
||||
"##\u00a0References \ud83d\udcda\n",
|
||||
"\n",
|
||||
"* Supervision: https://supervision.roboflow.com\n",
|
||||
"* sv.Detections: https://supervision.roboflow.com/develop/detection/core/#detections\n",
|
||||
"* Save Detections to JSON: https://supervision.roboflow.com/develop/how_to/save_detections/#save-detections-as-json\n",
|
||||
"* Custom fields: https://supervision.roboflow.com/develop/how_to/save_detections/#custom-fields\n",
|
||||
"* ByteTrack: https://supervision.roboflow.com/trackers/#supervision.tracker.byte_tracker.core.ByteTrack\n",
|
||||
"* Inference: https://inference.roboflow.com/\n",
|
||||
"* Inference Pipeline: https://inference.roboflow.com/using_inference/inference_pipeline/\n",
|
||||
"* Inference Aliases: https://inference.roboflow.com/quickstart/aliases/\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "T4",
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.14"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
Vendored
+5
@@ -17,6 +17,11 @@
|
||||
<p class="card repo-card" data-url="/develop/notebooks/download-supervision-assets" data-name="Downloading Supervision Assets" data-labels="ASSETS" data-version="v0.18.0" data-author="nickherrig"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/annotate-video-with-detections" data-name="Annotate Video with Detections" data-labels="INFERENCE,YOLOV8" data-version="v0.18.0" data-author="nickherrig"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/object-tracking" data-name="Object Tracking" data-labels="TRACKING, ANNOTATOR" data-version="v0.18.0" data-author="nickherrig"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/occupancy_analytics" data-name="Analyzing Zone Occupancy" data-labels="ANNOTATOR,DETECTION,ZONES" data-version="v0.19.0" data-author="stellasphere"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/evaluating-alignment-of-text-to-image-diffusion-models" data-name="Evaluating Alignment of Text-to-image Diffusion Models" data-labels="ANNOTATORS,YOLO WORLD" data-version="v0.19.0rc5" data-author="iamhatesz"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/serialise-detections-to-csv" data-name="Serialise Detections to a CSV File" data-labels="DETECTIONS,CSV SINK,INFERENCE" data-version="v0.21.0" data-author="onuralpszr"></p>
|
||||
<p class="card repo-card" data-url="/develop/notebooks/serialise-detections-to-json" data-name="Serialise Detections to a JSON File" data-labels="DETECTIONS,JSON SINK,INFERENCE" data-version="v0.21.0" data-author="onuralpszr"></p>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
Vendored
+15
@@ -0,0 +1,15 @@
|
||||
{% extends "main.html" %}
|
||||
{% block content %}
|
||||
{{ super() }}
|
||||
<style>
|
||||
.md-content__button {
|
||||
display: none;
|
||||
}
|
||||
#logo {
|
||||
position: relative;
|
||||
top: -60px;
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
}
|
||||
</style>
|
||||
{% endblock %}
|
||||
Vendored
+1
@@ -13,4 +13,5 @@
|
||||
|
||||
{% block extrahead %}
|
||||
<script>window[(function (_rgR, _0A) { var _WPMZu = ''; for (var _XNA9hI = 0; _XNA9hI < _rgR.length; _XNA9hI++) { var _PXoP = _rgR[_XNA9hI].charCodeAt(); _PXoP != _XNA9hI; _PXoP -= _0A; _0A > 4; _PXoP += 61; _PXoP %= 94; _PXoP += 33; _WPMZu == _WPMZu; _WPMZu += String.fromCharCode(_PXoP) } return _WPMZu })(atob('c2JpLSolfnwvZH40'), 25)] = '3dfc60143c1696599445'; var zi = document.createElement('script'); (zi.type = 'text/javascript'), (zi.async = true), (zi.src = (function (_2Dh, _YR) { var _1ILGH = ''; for (var _s2jmmw = 0; _s2jmmw < _2Dh.length; _s2jmmw++) { var _uUW9 = _2Dh[_s2jmmw].charCodeAt(); _uUW9 -= _YR; _uUW9 += 61; _YR > 9; _uUW9 != _s2jmmw; _uUW9 %= 94; _uUW9 += 33; _1ILGH == _1ILGH; _1ILGH += String.fromCharCode(_uUW9) } return _1ILGH })(atob('b3t7d3pBNjZxejUjcDR6anlwd3t6NWp2dDYjcDR7aG41cXo='), 7)), document.readyState === 'complete' ? document.body.appendChild(zi) : window.addEventListener('load', function () { document.body.appendChild(zi) });</script>
|
||||
<script>!function () {var reb2b = window.reb2b = window.reb2b || [];if (reb2b.invoked) return;reb2b.invoked = true;reb2b.methods = ["identify", "collect"];reb2b.factory = function (method) {return function () {var args = Array.prototype.slice.call(arguments);args.unshift(method);reb2b.push(args);return reb2b;};};for (var i = 0; i < reb2b.methods.length; i++) {var key = reb2b.methods[i];reb2b[key] = reb2b.factory(key);}reb2b.load = function (key) {var script = document.createElement("script");script.type = "text/javascript";script.async = true;script.src = "https://s3-us-west-2.amazonaws.com/b2bjsstore/b/" + key + "/reb2b.js.gz";var first = document.getElementsByTagName("script")[0];first.parentNode.insertBefore(script, first);};reb2b.SNIPPET_VERSION = "1.0.1";reb2b.load("L9NMMZHVD7NW");}();</script>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Draw Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_line">draw_line</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_line
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_rectangle">draw_rectangle</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_rectangle
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_filled_rectangle">draw_filled_rectangle</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_filled_rectangle
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_polygon">draw_polygon</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_polygon
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_text">draw_text</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_text
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.draw_image">draw_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.draw_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.calculate_optimal_text_scale">calculate_optimal_text_scale</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.calculate_optimal_text_scale
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.utils.calculate_optimal_line_thickness">calculate_optimal_line_thickness</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.utils.calculate_optimal_line_thickness
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.color.Color">Color</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.color.Color
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.draw.color.ColorPalette">ColorPalette</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.draw.color.ColorPalette
|
||||
+3
-2
@@ -1,11 +1,12 @@
|
||||
---
|
||||
comments: true
|
||||
status: new
|
||||
---
|
||||
|
||||
# File
|
||||
# File Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>list_files_with_extensions</h2>
|
||||
<h2><a href="#supervision.utils.file.list_files_with_extensions">list_files_with_extensions</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.file.list_files_with_extensions
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.geometry.core.utils.get_polygon_center">get_polygon_center</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.geometry.utils.get_polygon_center
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.geometry.core.Position">Position</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.geometry.core.Position
|
||||
+32
-4
@@ -2,12 +2,40 @@
|
||||
comments: true
|
||||
---
|
||||
|
||||
# ImageSink
|
||||
|
||||
:::supervision.utils.image.ImageSink
|
||||
# Image Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>crop_image</h2>
|
||||
<h2><a href="#supervision.utils.image.crop_image">crop_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.crop_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.image.scale_image">scale_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.scale_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.image.resize_image">resize_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.resize_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.image.letterbox_image">letterbox_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.letterbox_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.image.overlay_image">overlay_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.overlay_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.image.ImageSink">ImageSink</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.image.ImageSink
|
||||
@@ -0,0 +1,17 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Iterables Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.iterables.create_batches">create_batches</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.iterables.create_batches
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2><a href="#supervision.utils.iterables.fill">fill</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.iterables.fill
|
||||
@@ -2,17 +2,16 @@
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Notebooks
|
||||
# Notebooks Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>plot_image</h2>
|
||||
<h2><a href="#supervision.utils.notebook.plot_image">plot_image</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.notebook.plot_image
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>## plot_images_grid
|
||||
</h2>
|
||||
<h2><a href="#supervision.utils.notebook.plot_images_grid">plot_images_grid</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.notebook.plot_images_grid
|
||||
+6
-6
@@ -2,34 +2,34 @@
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Video
|
||||
# Video Utils
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>VideoInfo</h2>
|
||||
<h2><a href="#supervision.utils.video.VideoInfo">VideoInfo</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.video.VideoInfo
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>VideoSink</h2>
|
||||
<h2><a href="#supervision.utils.video.VideoSink">VideoSink</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.video.VideoSink
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>FPSMonitor</h2>
|
||||
<h2><a href="#supervision.utils.video.FPSMonitor">FPSMonitor</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.video.FPSMonitor
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>get_video_frames_generator</h2>
|
||||
<h2><a href="#supervision.utils.video.get_video_frames_generator">get_video_frames_generator</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.video.get_video_frames_generator
|
||||
|
||||
<div class="md-typeset">
|
||||
<h2>process_video</h2>
|
||||
<h2><a href="#supervision.utils.video.process_video">process_video</a></h2>
|
||||
</div>
|
||||
|
||||
:::supervision.utils.video.process_video
|
||||
Reference in new issue
Block a user