Can you please add support for basketball?

[gigasurgeon]

It would be very interesting to see a pipeline for basketball court and player tracking. Looking forward to it.

[hulkwork]

I think you will adapt for your use case by changing:

• Field detection part
• Change the homography matrix

[SkalskiP]

Hi @gigasurgeon and @hulkwork 👋🏻 Basketball is next on my TODO list. However, I don't know when I'll be able to get to it. Sports AI is just a side project, and I only work on it from time to time.

[jnebrera]

Hi @SkalskiP , I love the fact you are working on this project. I tried something similar for basketball. I have a labeled dataset of around 50K images with players and ball labelled. If you want, I could provide this dataset to you and your team to help in the effort.

In my case I planned to use a 180FOV camera seeing all the playfield, and then use some logic to decide were to point at to simulate panning.

[gigasurgeon]

@jnebrera I would be very interested in the dataset. I am working on a multitask basketball dense commentary generation. For that, the input features are player, basketball and court features. Currently, I do not have a good enough dataset for these classes. I was wondering if you could share the dataset with me. I would integrate the bbox prediction in one of the decoder heads.

[jnebrera]

Hi @gigasurgeon , yes, I can share the dataset with people with the commitment to release the model as open source "at least". I mean, if the company wants to build also a commercial product out of it is ok, BUT, an open source model (GPL, MIT, Apache) has to be provided with at least player and ball detection

[hulkwork]

• we probably need to annotate some dataset :) for the court (keypoint detections), and define the keypoints
• We can reuse the model for détecting players and seperate teams, at the first step
• For the ball, let's use what we have (retain if it not enough)

[SkalskiP]

@jnebrera wow 50K basketball images? 🔥 I'd love to work with that data! Do you have it in some standard format like COCO, YOLO, Pascal?

[SkalskiP]

@jnebrera We're currently using YOLOv8 (AGPL) for the soccer demo, but I'm keen on switching to RT-DETR (Apache-2.0) in the future. Check out my RT-DETR training tutorial here: https://blog.roboflow.com/train-rt-detr-custom-dataset-transformers/

With 50K images, we could really test the limits of RT-DETR! 🔥

[SkalskiP]

@hulkwork I already started to work on court keypoint detection dataset: https://universe.roboflow.com/roboflow-jvuqo/basketball-court-detection-2/dataset/1. But as you can see it steel needs a lot of love. 💜

[jnebrera]

Hi @hulkwork and @SkalskiP ,

Yes, field court is not labeled, and it would be quite complex too. I tried, but the lines are much harder to distinguish as this is amateur basketball and the court has millions of lines :D

Yes, images are properly labeled, I don't remember if Yolo or Pascal

Yes, I'm aware Yolo is AGPL, I also tried with it and with RTMDet (Apache)

As for court detection, if you use a fixed 180FOV camera, actually is not really needed as you can just "configure" the system defining the proper 4 field corners. Of course, much better to do it automatically, but if you think about it, 180FOV makes a lot of sense (Veo, Pixellot, all of those use it)

I also intended to do some pose estimation for those players that had the lower part of the bbox outside of the field, to see if they were actually out of the field (coach, public) or it was just a false positive

[SkalskiP]

@jnebrera, would it be possible for you to upload the dataset to Roboflow Universe?

[jnebrera]

I'm sorry but I can only share the dataset privately and the person receiving it has to maintain it the same way

BTW, at the end was 27K labeled images. I didn't remember I had like 20K still unlabeled

[SkalskiP]

@jnebrera, is there any chance you could share the subset privately so I can assess label quality?

[jnebrera]

Sure, my email is [email protected]. Please contact me there and we will arrange it

[luisrudge]

@jnebrera what camera are you using? I'm looking at adapting this project to use it in amateur soccer games (low quality recording). I'm using the Insta360 X3 to record my team's soccer matches but the quality is awful 😅

[jnebrera]

I was planning to use one of the dual lense Reolink cameras with stitching. Great quality, but 20fps

[luisrudge]

@jnebrera they look great actually. 20fps is a trade-off for sure, but maybe good enough considering the price? I guess if you count the fact that you need to buy an NVR to power it as well, it gets a bit expensive. If you could get the images directly via the ethernet cable, that'd be pretty nice.

[jnebrera]

NVR? The camera supports RTMP, as well as other IP protocols. You can get the video out live very easily

[luisrudge]

@jnebrera ohhh! TIL. thanks for sharing

[jnebrera]

Hi @SkalskiP , any progress with my dataset?