Donate object_store code from object_store_rs to arrow-rs

[alamb]

Draft until

• Ensure I can port CI from .circleci Add github test skeleton (NOT FOR MERGING) influxdata/object_store_rs#51 (will port as a follow on PR)
• IP Clearance: IP clearance for object_store_rs donation #2096
• Approved on mailing list: https://lists.apache.org/thread/5gfmvlf7lhs6lrcb0l8syq9s6dvs0mo8

Which issue does this PR close?

re #2030

Rationale for this change

See #2030

What changes are included in this PR?

These are kept in different commits for easier review:

• Copy in the code from object_store_rs at influxdata/object_store_rs@3c51870
• Update documentation for new home

Follow on Items:

• Port CI to github actions~ (I propose to do this in a follow on PR -- code is in Add github test skeleton (NOT FOR MERGING) influxdata/object_store_rs#51)
• File follow on ticket for ignored local::tests::bubble_up_io_errors

Are there any user-facing changes?

Not yet

[tustvold]

I wonder if we could preserve the history somehow, I'm mainly interested in not losing releases, but this would also help with in-flight PRs.

Perhaps you could add object_store_rs as a remote, add a commit moving it to a subdirectory, and then merge it into arrow-rs?

[alamb]

Update here: my git-fu is not strong enough to try and keep the history from object_store_rs -- @tustvold plans to give it a try. In the interim I will work on porting the tests to use github actions

[codecov-commenter]

Codecov Report

Merging #2081 (01f3908) into master (34f58f9) will decrease coverage by 0.78%.
The diff coverage is 55.08%.

@@            Coverage Diff             @@
##           master    #2081      +/-   ##
==========================================
- Coverage   83.71%   82.93%   -0.79%     
==========================================
  Files         225      237      +12     
  Lines       59632    61322    +1690     
==========================================
+ Hits        49923    50855     +932     
- Misses       9709    10467     +758     

Impacted Files	Coverage Δ
object_store/src/aws.rs	0.00% <0.00%> (ø)
object_store/src/azure.rs	0.00% <0.00%> (ø)
object_store/src/gcp.rs	0.00% <0.00%> (ø)
object_store/src/oauth.rs	0.00% <0.00%> (ø)
object_store/src/token.rs	0.00% <0.00%> (ø)
object_store/src/util.rs	50.00% <50.00%> (ø)
object_store/src/local.rs	82.88% <82.88%> (ø)
object_store/src/lib.rs	84.42% <84.42%> (ø)
object_store/src/path/mod.rs	90.90% <90.90%> (ø)
object_store/src/memory.rs	92.17% <92.17%> (ø)
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

[tustvold]

keep the history from object_store_rs

So unfortunately object_store_rs contains merge commits, so the only way to do this would break the linear history of arrow-rs, either by keeping the merge commits from object_store_rs/main or by adding a single merge commit. I therefore think its not worth it.

[alamb]

Update on test migration:

1. There appears to be one test that needs to be rewritten for github runners https://github.com/apache/arrow-rs/runs/7362535940?check_suite_focus=true (bubble_up_io_errors)
2. I have gotten the emulator tests running using github actions here: Add github test skeleton (NOT FOR MERGING) influxdata/object_store_rs#51 (it needs some polishing but I propose merging this PR first and then adding in object_store tests after the merge

[alamb]

There appears to be one test that needs to be rewritten for github runners https://github.com/apache/arrow-rs/runs/7362535940?check_suite_focus=true (bubble_up_io_errors)

@tustvold sorry to bother you -- but I was wondering if you had any ideas of ways to port the above test to run on github actions -- specifically it appears that setting the file mode to 0o000 doesn't actually generate an Error 🤔

[tustvold]

Probably the Github Actions runner is running as root, and therefore the permission failure doesn't occur. I'll rework the test

Edit: I can't think of an obvious way around this, I'd just remove the test - it isn't immensely valuable

[alamb]

Probably the Github Actions runner is running as root, and therefore the permission failure doesn't occur. I'll rework the test

I have #[ignored] the test in ed0f926 and will file a follow on ticket to fix it

[ursabot]

Benchmark runs are scheduled for baseline = add2649 and contender = 866f1a1. 866f1a1 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java