Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINTENANCE] Remove fluent partitioner methods from DataAssets #9517

Merged
merged 120 commits into from
Feb 28, 2024
Merged
Changes from 1 commit
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
c3969dd
wip
joshua-stauffer Feb 14, 2024
c9d892a
Merge branch 'develop' into f/v1-175/generic_partitioners
joshua-stauffer Feb 14, 2024
3cc0639
add generic partitioners
joshua-stauffer Feb 14, 2024
36239d9
move Partitioner out of typechecking
joshua-stauffer Feb 16, 2024
4854727
update import
joshua-stauffer Feb 16, 2024
19164a2
type
joshua-stauffer Feb 16, 2024
8bf119b
schema update
joshua-stauffer Feb 16, 2024
e695d19
add partitioner to BatchRequest
joshua-stauffer Feb 16, 2024
a167db1
update tests
joshua-stauffer Feb 16, 2024
50994e7
update test snapshot
joshua-stauffer Feb 16, 2024
e20b62e
add partitioner to DataAsset.build_batch_request
joshua-stauffer Feb 16, 2024
307263b
update type stub
joshua-stauffer Feb 16, 2024
5032352
update test snapshot
joshua-stauffer Feb 16, 2024
8193139
missed one
joshua-stauffer Feb 16, 2024
fa5cda8
Merge branch 'develop' into f/v1-175/add_partitioners_to_batch_request
joshua-stauffer Feb 16, 2024
9218fc9
add test
joshua-stauffer Feb 16, 2024
f8780c2
add partitioner map to base class
joshua-stauffer Feb 20, 2024
c7e9103
add partitioner maps
joshua-stauffer Feb 20, 2024
29e28af
rename spark partitioners
joshua-stauffer Feb 20, 2024
c0d2c1a
refactor sql partitioners
joshua-stauffer Feb 20, 2024
af5e0d9
rename spark partitioners
joshua-stauffer Feb 20, 2024
8795e5f
Merge branch 'develop' into f/v1-175/add_partitioners_to_batch_request
joshua-stauffer Feb 20, 2024
9d2bdf5
schema update
joshua-stauffer Feb 20, 2024
f9b2210
Merge branch 'f/v1-175/add_partitioners_to_batch_request' into f/v1-1…
joshua-stauffer Feb 20, 2024
89f7193
update per rename
joshua-stauffer Feb 20, 2024
4b405a0
Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc…
joshua-stauffer Feb 20, 2024
7bb9777
implement method to replace batch_options property
joshua-stauffer Feb 20, 2024
1c0c5d7
move partitioner resolve method to subclass
joshua-stauffer Feb 21, 2024
baff01e
typeguard
joshua-stauffer Feb 21, 2024
364dfaa
add override
joshua-stauffer Feb 21, 2024
f0d2f52
rename param
joshua-stauffer Feb 21, 2024
d626182
update types
joshua-stauffer Feb 21, 2024
a4e24b4
remove from interface
joshua-stauffer Feb 21, 2024
269089e
add types to subclass
joshua-stauffer Feb 21, 2024
f00a1fb
tweak type
joshua-stauffer Feb 21, 2024
dbd95d0
revert
joshua-stauffer Feb 21, 2024
1004b30
types
joshua-stauffer Feb 21, 2024
8623707
move type out of typechecking
joshua-stauffer Feb 21, 2024
ac3ee51
move import
joshua-stauffer Feb 21, 2024
85f072a
try type as function
joshua-stauffer Feb 21, 2024
9ed5cee
revert
joshua-stauffer Feb 21, 2024
c2ba4c7
hacky type workaround
joshua-stauffer Feb 21, 2024
fbf7657
use Type
joshua-stauffer Feb 21, 2024
abc76d5
spark: use batch request partitioner
joshua-stauffer Feb 21, 2024
0cf9e3a
sql: use batch request partitioner
joshua-stauffer Feb 21, 2024
de6fc0c
add sqlite partitioners
joshua-stauffer Feb 21, 2024
454aef8
fix sqlite tests
joshua-stauffer Feb 22, 2024
7a9fafa
use get_batch_request_options instead of batch_request_options property
joshua-stauffer Feb 22, 2024
6edf63f
wip
joshua-stauffer Feb 22, 2024
7da1552
rename method
joshua-stauffer Feb 22, 2024
aa244ab
sqlite tests passing
joshua-stauffer Feb 22, 2024
d0b3961
update signature
joshua-stauffer Feb 22, 2024
ab1149f
update types
joshua-stauffer Feb 22, 2024
e2e4d16
move ConvertedDatetime partitioner into sql
joshua-stauffer Feb 22, 2024
d801d17
ensure sqlite assets have access to correct partitioner map
joshua-stauffer Feb 22, 2024
f97c441
schema update
joshua-stauffer Feb 22, 2024
d88a277
Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc…
joshua-stauffer Feb 22, 2024
d0ff1ca
schema update
joshua-stauffer Feb 22, 2024
b924f97
refactor partitioner to batch request in conftest
joshua-stauffer Feb 23, 2024
7b7247f
update integration tests to use partitioner in batch request
joshua-stauffer Feb 23, 2024
3362915
remove deprecated partitioner
joshua-stauffer Feb 23, 2024
7611ecb
Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc…
joshua-stauffer Feb 23, 2024
46a5c3e
remove sql partitioner_year
joshua-stauffer Feb 23, 2024
ff5abdf
schema update
joshua-stauffer Feb 23, 2024
b38fa01
remove sql add_partitioner_year_and_month
joshua-stauffer Feb 23, 2024
4953def
remove sql add_partitioner_year_and_month_and_day
joshua-stauffer Feb 23, 2024
5cf5c9e
remove sql add_partitioner_datetime_part
joshua-stauffer Feb 23, 2024
519867e
remove sql add_partitioner_column_value
joshua-stauffer Feb 23, 2024
6137d92
update integration tests
joshua-stauffer Feb 23, 2024
ec0e7f1
remove sql add_partitioner_divided_integer
joshua-stauffer Feb 23, 2024
dc32b3b
remove sql add_partitioner_mod_integer
joshua-stauffer Feb 23, 2024
5b188d9
remove sql add_partitioner_multi_column_values
joshua-stauffer Feb 23, 2024
795c831
remove remaining helper methods
joshua-stauffer Feb 23, 2024
3e708f8
remove sql test_partitioner
joshua-stauffer Feb 23, 2024
2509852
remove references to batch_request_options
joshua-stauffer Feb 23, 2024
1ae3497
update viral snippet
joshua-stauffer Feb 23, 2024
990cb11
remove sql batch_request_options
joshua-stauffer Feb 23, 2024
d91e36d
update tests related to fixture
joshua-stauffer Feb 23, 2024
6c964ad
update postgresql tests
joshua-stauffer Feb 23, 2024
e7d28e3
update snippets
joshua-stauffer Feb 23, 2024
b19afcf
Merge branch 'f/v1-175/asset_uses_partitioner_from_batch_request' int…
joshua-stauffer Feb 23, 2024
e6e777c
add partitioners to public api
joshua-stauffer Feb 23, 2024
175814b
public api
joshua-stauffer Feb 23, 2024
53d8c89
schema update
joshua-stauffer Feb 23, 2024
4d3b6ee
update api excludes
joshua-stauffer Feb 23, 2024
344bedc
Merge branch 'f/v1-175/asset_uses_partitioner_from_batch_request' int…
joshua-stauffer Feb 23, 2024
4eab300
fix doc snippet
joshua-stauffer Feb 23, 2024
4b0651e
remove assert
joshua-stauffer Feb 26, 2024
43888ed
remove comment
joshua-stauffer Feb 26, 2024
9d42d57
remove batch config option keys method from batch config
joshua-stauffer Feb 26, 2024
cb0987c
Merge branch 'f/v1-175/asset_uses_partitioner_from_batch_request' int…
joshua-stauffer Feb 26, 2024
0b63506
Merge branch 'develop' into m/v1-175/remove_legacy_partition_methods
joshua-stauffer Feb 26, 2024
3a4b906
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 26, 2024
52e4f63
update test
joshua-stauffer Feb 26, 2024
dbd18ac
remove batch_config_options
joshua-stauffer Feb 26, 2024
3dac9c9
update batch config persistence behavior
joshua-stauffer Feb 26, 2024
9c1134e
ensure partitioner is an optional param
joshua-stauffer Feb 26, 2024
8ad0efc
update postgresql tests
joshua-stauffer Feb 27, 2024
17a32b7
update docs
joshua-stauffer Feb 27, 2024
c6f3772
remove spark add_partitioner_year
joshua-stauffer Feb 27, 2024
f34d8d7
remove spark add_partitioner_year_and_month
joshua-stauffer Feb 27, 2024
854d523
remove spark add_partitioner_year_and_month_and_day
joshua-stauffer Feb 27, 2024
c7a899b
remove spark add_partitioner_datetime_part
joshua-stauffer Feb 27, 2024
d79245c
remove spark add_partitioner_column_value
joshua-stauffer Feb 27, 2024
b666761
remove spark add_partitioner_divided_integer
joshua-stauffer Feb 27, 2024
82ac959
remove spark add_partitioner_mod_integer
joshua-stauffer Feb 27, 2024
b020097
remove spark add_partitioner_multi_column_values
joshua-stauffer Feb 27, 2024
a3bc3b5
remove _add_partitioner
joshua-stauffer Feb 27, 2024
e8954fe
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 27, 2024
8b237f6
Merge branch 'develop' into m/v1-175/remove_legacy_partition_methods
joshua-stauffer Feb 27, 2024
31b6674
remove type ignores
joshua-stauffer Feb 27, 2024
b4be3df
add return type
joshua-stauffer Feb 28, 2024
ae5bb0d
remove sqlite add_partitioner method
joshua-stauffer Feb 28, 2024
9198fe6
remove DataAsset.partitioner and update tests
joshua-stauffer Feb 28, 2024
e9d1fde
Merge branch 'develop' into m/v1-175/remove_legacy_partition_methods
joshua-stauffer Feb 28, 2024
18dec3f
update yaml
joshua-stauffer Feb 28, 2024
212de7e
remove stray reference to DataAsset.partitioner
joshua-stauffer Feb 28, 2024
9d3bfd5
update unit test
joshua-stauffer Feb 28, 2024
92b7b0f
up schema
joshua-stauffer Feb 28, 2024
4720bd0
add back removed test
joshua-stauffer Feb 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'develop' into m/v1-175/remove_legacy_partition_methods
  • Loading branch information
joshua-stauffer committed Feb 27, 2024
commit 8b237f64893453bd0a16f29b8cf7f13a188c544e

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.