Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

V1.3 release #301

Merged
merged 490 commits into from
Dec 2, 2022
Merged

V1.3 release #301

merged 490 commits into from
Dec 2, 2022

Conversation

catherinebirney
Copy link
Collaborator

@catherinebirney catherinebirney commented Dec 1, 2022 •
edited
Loading

Major Updates:

  • Option to append material codes to end of sector code in FBS
  • New waste-related FBAs/FBSs, and 7/8 digit sector codes
  • New food waste and concrete waste specific FBS
  • New data visualization functions: Sankey, stackedBarChart
  • Updates to methodology since v1.2.4 release changed the results for: CAP_HAP_national_2017, CNHW_national_2014, Land_national_2012, Water_national_2010_m2, Water_national_2015_m2, Water_national_2015_m3
  • option to return an FBS with any mix of sector lengths, SPB and SCB columns no longer need to have matching sector lengths
  • option for multiplication as allocation method
  • update allocation methodology so if the allocation dataset is at a more aggregated geoscale than the primary FBA, that the primary df is not aggregated to match the allocation geoscale. Instead the more aggregated allocation dataset merges with the primary FBA on all related less aggregated geoscales
  • Changes to github actions to address memory issues
  • in equal_allocation() aggregate column before allocating to child naics - this impacts some NAICS6 results
  • option to retain activity names in a primary data source after calling on those activities in an FBS, so the activity names can be used again
  • requires plotly, kaleido

Minor Updates:

  • new external_paths.env file to store local file paths
  • rename load_api_key to load_env_file_key(), used for api keys and external_paths.env
  • getFlowByActivity accepts flowclass of class or list
  • new sector codes: S00203 - other state and local government enterprises, F040 - exports of goods and services, F050 - imports of goods and services, new 7/8 digit sectors for waste sectors (5622121, 5622191, 5622192, 5629201, 5629202, 5629203)
  • New required columns in FBS: ProducedBySectorType, ConsumedBySectorType, AttributionSources (primary attribution source name)
  • new selection_fields FBS parameter to subset FBA using column names/values
  • new VisualizationEssentials.csv with standard colors for graphing sectors
  • new csv "sector_2012_names" which includes additional names beyond official NAICS

New/Modified FBA:

  • expand epa_cddpath sector mapping
  • EPA_FactsAndFigures, EPA_REI, EPA_WARMer, EPA_WFR, Census_ASM, EPA_CDDPath (2018), EIA_AEO, EIA_SEDS

New/Modified FBS:

  • CNHW_national_2018, Employment_national (2012, 2014, 2015), Food_waste_national_2018 (m1, m2), GHG_national_m1 (2016, 2017, 2018, 2019), REI_waste_national_2012

catherinebirney and others added 30 commits September 7, 2022 12:37
@catherinebirney
in compare fbs fxn, ignore columns for now as not in previous FBS dfs
271700c
@catherinebirney
drop unused parameter requirement causing error for tri_dmr_state
0f7eaba
@catherinebirney
add example sankey
8e7f1ad
@catherinebirney
add abrev name to sector color csv
7c745d9
@catherinebirney
drop vis dic since it is now csv
d5e7f09
@catherinebirney
add sector abbreviations for SPB, drop units
f611ece
@catherinebirney
initial code to add waste treatment activiites to food waste m1
40506d1
@catherinebirney
Merge branch 'develop' into waste
069eb3b
@catherinebirney
Merge branch 'waste' into DataVis
ca78597
@catherinebirney
grouped stacked barchart working for horizontal graphs
32919ab
@catherinebirney
grouped stacked barchart working for vertical graphs
f5fdc61
@catherinebirney
update stacked barplots - wrap label texts, update axis labels, de-du…
f517499 
…plicate legend, dave file locally
@catherinebirney
update cdd concrete fbs so target subset level is naics7
1ec14ce
@catherinebirney
Merge branch 'waste' into DataVis
507a03d
@catherinebirney
update tabula import to use tabula-py
28d9752
@catherinebirney
Merge branch 'waste' into DataVis
ff2df63
@catherinebirney
update params
6b2629e
@catherinebirney
modify row/column assignment within for loop
7a87e86
@catherinebirney
barchart - convert units and modify output sizing
74c9284
@bl-young
new mapping for CDD_concrete unique from CDDPath
3ad51ed
@bl-young
revise ACB mappings to reflect waste management naics
72242e9
@catherinebirney
aggregate cddpath after resetting APB col
4bfade9
@catherinebirney
move df aggregation from keep_activity_consumed_by() to equal_allocat…
89d7292 
…ion()
@bl-young
fix missing landfill cover
59794bd
@catherinebirney
initial rei primary factors fba code
b60e88c
@catherinebirney
produce primary factors and waste flows with one FBA call - incomplete
85c7b07
@catherinebirney
add treated waste sectors to wfr crosswalk
a56e9c3
@catherinebirney
add treated waste commodities to food waste m1
aa5b788
@catherinebirney
Merge branch 'waste' into food_waste
8a57276
@catherinebirney
add treated waste to food waste fbs m1
f24465f
@catherinebirney
add example stacked barcharts for ghg FBS
4dcf5e2
@catherinebirney
revert 'sector_2012_names' back to 'naics_2012_names'
ce4b65a
@catherinebirney
Keep new "sector_2012_names" because includes new, unofficial sector …
c68548a 
…names used in graphing

This reverts commit ce4b65a.
@catherinebirney
in compare_fbs_results, if column does not exist in merge_col, ignore…
a3052ee 
… error - missing columns caused by new cols from DataVis branch
@catherinebirney
also ignore 'allocationsources' in compare_fbs_results - can remove l…
3172bb9 
…ine of code once datavis FBS are on data commons
@catherinebirney
Merge pull request #297 from USEPA/ghg_fbs
cb5f3d2 
add ghg national fbs m1 2016, 2018, 2019
@catherinebirney
add kaleido to package requirements/setup and alphabetize packages
1a507c8
@catherinebirney
rename add_column_of_allocation_sources() to add_attribution_sources_…
5010cfd 
…col(), rename AllocationSourcs col to AttributionSources, and change the fxn so only adds primary attribution source to col, excluding secondary sources
@catherinebirney
correct typo
dd7a0ad
@catherinebirney
drop kaleido==0.1.0.post1 requirement because that version only works…
4241a59 
… on windows
@catherinebirney
Revert "drop kaleido==0.1.0.post1 requirement because that version on…
174a0e5 
…ly works on windows"

This reverts commit 4241a59.
@catherinebirney
set kaleido requirements based on os
e0aadb4
@catherinebirney
do not need to explicitly import kaleido as plotly automatically uses…
eb46af1 
… kaleido to write static images if kaleido installed
@catherinebirney
update kaleido use explanation
ff52189
@catherinebirney
Merge pull request #300 from USEPA/develop
7f275b4 
update v1.3-release with develop
@catherinebirney
update package version and drop develop from package requirements
fb50ef6
@catherinebirney
option to specify name of the FlowAmount col in aggregator()
5237d68
@catherinebirney
in compare_fbs_results() aggregate dfs before merge to drop duplicate…
393c7fc 
… rows caused by dropping metasources/attributionsources cols
@catherinebirney
Merge branch 'develop' into DataVis
0d7bec0
@catherinebirney
option to pass filename in sankey and stackedbarchart functions
1dc16ad
@catherinebirney
Merge pull request #270 from USEPA/DataVis
6183153 
Data vis
@catherinebirney
Merge branch 'develop' into v1.3-release
509762e 
# Conflicts:
#	requirements.txt
#	setup.py
@catherinebirney catherinebirney marked this pull request as draft December 2, 2022 14:46
@catherinebirney catherinebirney marked this pull request as ready for review December 2, 2022 14:46
@catherinebirney
Copy link
Collaborator Author

@bl-young Changes to FBS are what we expect. If you agree with results and the merge notes I wrote out above, I will merge this PR and publish the v1.3.0 release

@catherinebirney catherinebirney merged commit 41206c8 into master Dec 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bl-young bl-young bl-young approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@catherinebirney @bl-young @andychase