Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new example for video feature vector input #293

Open
wants to merge 63 commits into
base: main
Choose a base branch
from

Conversation

xianyuanliu
Copy link
Member

@xianyuanliu xianyuanliu commented Jan 26, 2022

Description

This PR follows PR #291 and #292, upgrading DA Trainers for feature vector input. Trainers on video image input and video feature vector input have two main differences below:

  1. Three modality choices in image input: RGB, Flow and joint (RGB+Flow), due to no Audio available now; Five in feature vector: RGB, Flow, Audio, joint and all (RGB+Flow+Audio).
  2. One class type in image input: verb; Two in feature: verb, verb+noun.

The main changes are below:

  1. Add Audio branch into the pipeline.
  2. Update optimizer work with SGD and ADAM.
  3. Add adaptive choice into source/target dataset sampling.
  4. Add JSON logger for EPIC UDA challenge.
  5. Reduce dummy network dimension in test_video_domain_adapter.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • In-line docstrings updated.
  • Source for documentation at docs manually updated for new API.

@xianyuanliu xianyuanliu added work-in-progress Work in progress that should NOT be merged new feature New feature/module (including request) labels Jan 26, 2022
@github-actions github-actions bot added this to In progress in v0.1.0 Jan 26, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jan 27, 2022

Codecov Report

Merging #293 (2f7770f) into main (e42b4ec) will decrease coverage by 0.67%.
The diff coverage is 90.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #293      +/-   ##
==========================================
- Coverage   92.98%   92.31%   -0.68%     
==========================================
  Files          45       45              
  Lines        4635     5163     +528     
==========================================
+ Hits         4310     4766     +456     
- Misses        325      397      +72     
Impacted Files Coverage Δ
kale/pipeline/domain_adapter.py 94.08% <ø> (ø)
kale/loaddata/multi_domain.py 89.50% <53.84%> (-3.14%) ⬇️
kale/loaddata/video_multi_domain.py 72.12% <67.85%> (-2.69%) ⬇️
kale/loaddata/videos.py 88.53% <90.00%> (+1.16%) ⬆️
kale/pipeline/video_domain_adapter.py 91.70% <91.01%> (-5.24%) ⬇️
kale/embed/video_feature_extractor.py 95.34% <93.10%> (-1.27%) ⬇️
kale/loaddata/video_access.py 96.55% <94.44%> (-1.37%) ⬇️
kale/embed/attention_cnn.py 98.03% <98.38%> (+0.53%) ⬆️
kale/embed/video_selayer.py 97.32% <100.00%> (+0.32%) ⬆️
kale/loaddata/dataset_access.py 83.33% <100.00%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e42b4ec...2f7770f. Read the comment docs.

@xianyuanliu xianyuanliu removed the work-in-progress Work in progress that should NOT be merged label Jan 28, 2022
@xianyuanliu xianyuanliu marked this pull request as ready for review January 29, 2022 04:23
@haipinglu haipinglu removed this from In progress in v0.1.0 Aug 11, 2022
@haipinglu haipinglu added this to In progress in v0.2.0 via automation Aug 11, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2022

This pull request has been automatically marked as stale due to lack of activity.

@github-actions github-actions bot added the Stale label Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature/module (including request) Stale
Projects
Status: In progress
v0.2.0
In progress
Development

Successfully merging this pull request may close these issues.

None yet

2 participants