Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/reduce-event-gather-complexity #232

Merged
merged 6 commits into from
May 6, 2023

Conversation

evamaxfield
Copy link
Member

Had parts of this code sitting around. Finished it up in ~30 minutes. Need to test on a dev instance but pretty confident it will work.

Main change is resolving #230 -- moving from processing all sessions in a single flow to producing multiple flows, one for each session. I haven't touched any multiprocessing just yet to be safe.

@evamaxfield
Copy link
Member Author

evamaxfield commented May 6, 2023

Heading out for the night, might get a chance to test this tomorrow. If it works on a dev instance will fast merge and deploy

@codecov
Copy link

codecov bot commented May 6, 2023

Codecov Report

Merging #232 (3169ec3) into main (ad09d9c) will decrease coverage by 0.39%.
The diff coverage is 39.53%.

❗ Current head 3169ec3 differs from pull request most recent head 95a29a7. Consider uploading reports for the commit 95a29a7 to get more accurate results

@@            Coverage Diff             @@
##             main     #232      +/-   ##
==========================================
- Coverage   72.01%   71.62%   -0.39%     
==========================================
  Files          50       50              
  Lines        3312     3330      +18     
==========================================
  Hits         2385     2385              
- Misses        927      945      +18     
Impacted Files Coverage Δ
cdp_backend/bin/process_special_event.py 0.00% <0.00%> (ø)
cdp_backend/bin/run_cdp_event_gather.py 0.00% <0.00%> (ø)
cdp_backend/tests/utils/test_file_utils.py 100.00% <ø> (ø)
cdp_backend/pipeline/event_gather_pipeline.py 82.20% <86.66%> (+0.05%) ⬆️
...ckend/tests/pipeline/test_event_gather_pipeline.py 98.36% <100.00%> (+0.01%) ⬆️
cdp_backend/utils/file_utils.py 88.29% <100.00%> (ø)

@evamaxfield evamaxfield merged commit 6c1c1ff into main May 6, 2023
6 checks passed
@evamaxfield evamaxfield deleted the feature/reduce-event-gather-pipeline-complextity branch May 6, 2023 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant