Skip to content
/ whacc Public

semi-automatic and customizable pipeline for classifying whiskers contacting objects. WhACC uses a ResNetV2-Light GBM hybrid model to predict touch times in high-speed video. Included is a tracker to extract images from video, a GUI to label extracted data, and a retrain interface to customize the light GBM model head to your data.

License

Notifications You must be signed in to change notification settings

hireslab/whacc

Repository files navigation


WhACC is a tool for automated touched image classification.

Many neuroscience labs (e.g. Hires Lab) use tasks that involve whisker active touch against thin movable poles to study diverse questions of sensory and motor coding. Since neurons operate at temporal resolutions of milliseconds, determining precise whisker contact periods is essential. Yet, accurately classifying the precise moment of touch is time-consuming and labor intensive.

Model Card

Data Card


Single example trial lasting 4 seconds. Example video (left) along with whisker traces, decomposed components, and spikes recorded from L5 (right). How do we identify the precise millisecond frame when touch occurs?


Original 2048 output features extracted from the penultimate layer of the initial ResNet50 V2 model, clustered for emphasize

Flow diagram of WhACC video pre-processing and design implementation


Touch frame scoring and variation in human curation


Data selection and model performance


Feature engineering and selection


WhACC shows expert human level performance


WhACC can be retrained on a small subset to account for data drift over time or different datasets (see GUI below)


WhACC GUI: used to curate automatically selected subset of data for optimal performance


Use left and right arrows to move through images, use up to label as touch (green) and down to label as not-touch (red)


Code contributors:

WhACC code and software was originally developed by Phillip Maire and Jonathan Cheung in the laboratory of Samuel Andrew Hires.

About

semi-automatic and customizable pipeline for classifying whiskers contacting objects. WhACC uses a ResNetV2-Light GBM hybrid model to predict touch times in high-speed video. Included is a tracker to extract images from video, a GUI to label extracted data, and a retrain interface to customize the light GBM model head to your data.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published