Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
👋 Hello here
In many real-world scenarios, training a detection model, deploying it, and getting predictions is only one half of the job. Processing of the predictions is the other half of data scientists' work, and Supervision is the only tool to help do that in object detection.
However, thinking about my previous jobs and what I needed, I realized that there was a part of post-processing that was not covered by Supervision: looking for patterns. We can think of it as a regex for object detection: Given a set of rules, give me the set of boxes that respect these rules.
For instance in this image with predictions:
![g15](https://private-user-images.githubusercontent.com/14785566/310940891-df490464-eb30-404b-8f9e-cb9be4320140.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM5NzA5MTgsIm5iZiI6MTcyMzk3MDYxOCwicGF0aCI6Ii8xNDc4NTU2Ni8zMTA5NDA4OTEtZGY0OTA0NjQtZWIzMC00MDRiLThmOWUtY2I5YmU0MzIwMTQwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE4VDA4NDMzOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU5N2JmOWU2ZGM4N2M3ZGNjY2Q1ZDVkNWFmMzMxZTBhZmIzNTBjMDA3NDQyNjZiYzNlZDZlYjYzZWE2ODUxMDcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.inn-FbBiF8tSOeJfuOYNUB9tcl1QeVH_K67_LjJIFXI)
That we obtained with:
We could add a pattern to find an individual with their bike. The rule to identify this pattern (the regex) would be:
matches
is a list of detections (List[sv.Detections]
) that match the pattern. Once displayed, the 4 first matches from the list look like this:For each match, an additional
data
attribute named "match_name" is added to thesv.Detection
to specify to which detection corresponds each object of the pattern ("Bike" and "Rider" in our case).Additional notes
1 - I added a little sugar on top of the feature. It is possible to define a value instead of a function. In that case, it will ensure that the arguments match exactly the value you provide.
2 - The arguments you can give to the function directly come from
sv.Detections
. The complete list is:xyxy
,mask
,class_id
,confidence
, ortracker_id
Position
enum stringsdata
attribute of your detections for more custom use cases3 - It we want to continue this feature, it would be nice to provide a small bank of function that the user can use out of the box for bounding boxes (
iou
,included_in
, etc.) and segmentation (next_to
,area_are_touching
, etc.)Type of change
Please delete options that are not relevant.
How has this change been tested, please provide a testcase or example of how you tested the change?
I added functional tests.
Docs