Skip to content

Latest commit

 

History

History
243 lines (178 loc) · 13.1 KB

22.md

File metadata and controls

243 lines (178 loc) · 13.1 KB
Raw
title
2022

ra

Rapid Annotator @ 2022

Contents

Introduction

Red Hen’s Rapid Annotator provides a platform to users to annotate large chunks of data in a short span of time and with least possible efforts. It provides the features of annotating images/videos/audios/text during collaborative situation also. With Red Hen Lab’s we will try to enable users to visualize the progress of each annotator separately and annotators can notify experimenter when the annotation is finished to make the annotation work more efficient.
In Red Hen Lab’s Rapid Annotator we try to enable researchers worldwide to annotate large chunks of data in a very short period of time with least effort possible and try to get started with minimal training. I have been continuing working on Red Hen Lab's Rapid Annotator after work done by Gulshan Kumar and Vaibhav Gubta at GSoC's 20, 19 and 18.

Terminologies

  • RA stands for Rapid Annotator
  • PR stands for pull request
  • BG stands for bug, usualy will mention an issue

Built with

  • Python [Flask]
  • Javascript
  • HTML
  • CSS

Important links

Pull requests list

  1. Experiment updated configuration (for new added text-fields for selected labels) #63

    Checkbox for each annotation level to configure whether you want to allow the text-field for each selected text box or not through the annotation level settings from the experiment's labels page

    image

    In case of making it on

    Text fields next to the selected label image

    In case of making it off

    No text fields next to the selected label image

    Demo

    labelconfig2

  2. Allowing users to import their own annotations levels without making it global (self-access) #64

    Description

    Allowing the experiments' owners to import their annotation levels without doing the over-head step [make it global, import it then make it private again]

    Demo

    users-own-levels-import

  3. Sharing experiment's annotation levels to certain users #65

    Description

    Allowing sharing the annotation levels of an experiment to certain users selection from the settings page

    Screenshot (Experiment settings page)

    image

    Demo

    2share-levels

  4. Elan experiment type #66

    Description

    Elan experiment: is a new category of experiments (extension to video experiment) that is needed to be added to rapid annotator that allows multiple and infinite count of annotations for each level. New analogies (for elan experiment):

    • Level: Tier
    • Label: (Annotation) Text

    The structure of each annotation entry will be as the following (description of database changes): Note: Database schema updated image is added in docs folder ElanAnnotation:

    • id - Int (auto increment)
    • experimentId - the experiment is being in the process
    • fileId - the file is being annotated
    • data - json ; that is the point to be explained

    Review of what is the replacement: each selected label was recorded as a single record in AnnotationInfo table. For elan it is instead of keeping a single record for each entry (as they are infinite; not limited to the count of labels inside each level) it was better to use a json type So far, data is a json type contains the following:

    {
"levelId01": [{"startTime": 0.01, "endTime": 0.036, "text": "Anythingcan be here"}]
}

    It is an object contains (levels/tiers ids of the experiment as keys and for each tierId it contains an array which is the content of annotations) Sample view explained image that image is represneted as the following:

    {
  "34": [
    {
      "text": "label 1",
      "endTime": 6.5,
      "startTime": 0
    },
    {
      "text": "label 3",
      "endTime": 19.25,
      "startTime": 10.0625
    }
  ],
  "35": [
    {
      "text": "dsa",
      "endTime": 15,
      "startTime": 2.4375
    }
  ]

    Results is exported in 2 ways

    1. All results of the experiment at once in a tabular format contains the following columns (['File Name', 'Annotator', 'Tier Name', 'Annotation Text', 'Start Time (sec)', 'End Time (sec)']) as (csv, xlsx)
    2. Export a .eaf (elan file format) of a specific user's annotation for a specific file

    User interface changes:

    • When creating a new experiment, you will find a new type image
    • At view results page you will find a new column that allows you to download a .eaf file for each annotation of the selected user and clicked file image
    • New experiment interface (elan-similar) localhost_5000_elan_specificAnnotation_1_17_49 (1)
    • Key bindings image

    2keybind

    • Add/Edit annotation form image

    • Add/Edit annotation form Demo alltogether

    • Timeline image

    • Timeline Demo timeline demo

    Functions

    • Zoom in/out
    • Scroll (zoom mode or scroll up/down mode)
    • Hide other annotations of a track when on is selected, or show all
    • Control video buttons

    Complete demo (video) for annotation to export results!

    • on Youtube

    • The annotation tiers became accessible through numbers as shortcut keys (i.e., first tier can be selected be pressing '1' and second through '2' and etc.).

    tiershortcut

    • Exporting results of grouped annotators of an elan file has been added also. groupedresults

  5. Elan labels shortcut keys added and fixed an issue #67

    • Added the shortcut key of each label below the label image

    • Fixed the labels' data-list after adding the numbers shortcut keys for the tiers' selection (the labels options were not being changed)

  6. Enable/Disable advanced options in the experiment and fixed specific view #68

    That PR contains two main additions

    • A new configuration to the experiment to control whether you want to add the new features those added last year in that PR #60 so, at the adding new experiment we get a new field for the experiment's table called advanced annotation and at the creation of the experiment it appears as a checkbox.

    image

    • At viewing a specific entry of the experiment files (the advanced features above were missing from that view, so they have been added and are being loaded and edited also from specific annotation view) (Example of before and after at an image category experiment)
    After Before
    example_after example_before

Challenges

  • The most challanging part was my availability schedule which is solved by GSOC's extension of the timeline (accepted by the mentors; I would like to say thanks)
  • The (ELAN software simple version) software is supposed to be mimicked in the Rapid Annotator to contain a video timeline and tracks annotation was the hardest part in the project; it is done and reached a good state by the helping of mentors and open-source community as well was a part of the solution because I found an inital state that I can start from instead of starting from scratch and lose time that can be used in a more useful way

Conclusion

  • Experiment configuration whether you want to allow the text-field for each selected text box or not through the annotation level settings from the experiment's labels page is added
  • Sharing experiment's annotation levels to certain users feature
  • Allowing the experiments' owners to import their annotation levels without doing the over-head step [make it global, import it then make it private again]
  • ELAN experiment type to the RapidAnnotator is added as a new experiment category (ELAN software simple version)
  • Ability to use the exported results of elan experiment from rapid annotator to elan software (compatible elan file format)
  • Enable/Disable advanced options in the experiment and fixed specific view A new configuration to the experiment to control whether you want to add the new features those added last year in that PR Annotations adds-on (coordinates feature) #60 so, at the adding new experiment we get a new field for the experiment's table called advanced annotation and at the creation of the experiment it appears as a checkbox.
  • At viewing a specific entry of the experiment files (the advanced features above were missing from that view, so they have been added and are being loaded and edited also from specific annotation view)