website | [arxiv] | [CVF]

The idea behind our work is to tackle the high variance of error that is ignored when considering de facto statistical performance measures like (MSE,MAE) for performance evaluation in the crowd counting domain. Our recipe involves finding strata that are optimal in a Bayesian sense and later systematically modifying the standard crowd counting pipeline to incorporate decrease of variance at each step.

The website contains interactive plotly plots, which bring out the issue with using MAE/MSE as evaluation metrics. Additionally, it contains the plots that enable inspecting the performance at a bin-level.

In our paradigm the first step is to create strata over the count range to enable balanced data splits. We recommend to divide the entire range of counts into strata which are optimal in Bayesian sense, and later sample images that belong to these strata. The procedure for data split generation is provided in the folder named binning.

Next step in our paradigm is to sample from these bins. We exeprimented with two differnt ways of choosing the bin, they are :

    1. Round Robin sampling
    2. Random sampling

For a simple visualisation of the procedure you can refer to the folder named sampling. In the same folder, we present a demo on DM Count network about procedure to include the sampling into any network.

The next step of the procedure is to include strata aware optimization into any network. We add this loss as an additive loss along with a scaleable hyper parameter (λ₁ in Sec 3.4) to the original network's loss function. The idea behind adoption of strata aware optimization is to reduce variance. A pyTorch implementation of that optimization is provided in this folder named optimization.

The final step in our paradigm is to have a better represented evaulation metric. The generally used MAE conceals standard deviation, to bring out this we evaluate the performance at a strata-level (mean and std) and a pooled mean and std. The folder evaluation consists of a notebook to do the same.

To include your network in our work please create a pull request with the name <your_network_name>. Fill up the details of your network in the following format, and add it into the description:

Network name:
Conference or arxiv link:
Github repo:
Contact email:

Attach a file with the name <your_network_name_dataset_name>.txt, replace your_network_name with your networks name and dataset_name with the dataset name on which results of the file correspond to. The file should contain row-wise information in the following format:

image_name_1 , predicted_1
image_name_2 , predicted_2
.
.
.
image_name_n , predicted_n

Here image_name_i can be a string or an int that refers to the input image name to the network and predicted_i refers to the count predicted by your network from the image_name_i.

If you are using our work, Please cite us at :

@inproceedings{10.1145/3474085.3475522,
author = {Sravya Vardhani Shivapuja, Mansi Pradeep Khamkar, Divij Bajaj, Ganesh Ramakrishnan, Ravi Kiran Sarvadevabhatla},
title = {Wisdom of (Binned) Crowds: A Bayesian Stratification Paradigm
for Crowd Counting
},
booktitle = {Proceedings of the 2021 ACM Conference on Multimedia},
year = {2021},
location = {Virtual Event, China},
publisher = {ACM},
address = {China},
}