Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

object_store: add Path::from_url_path #3663

Merged

Conversation

jychen7
Copy link
Contributor

@jychen7 jychen7 commented Feb 5, 2023

Which issue does this PR close?

Closes #3651

Rationale for this change

#3651 (comment)

This would simplify user usage of Path, e.g.

What changes are included in this PR?

Implement new method from_url_path

Are there any user-facing changes?

New method is provided, no impact to existing methods

@github-actions github-actions bot added the object-store Object Store Interface label Feb 5, 2023
Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, mostly just some additional test suggestions

object_store/src/path/mod.rs Outdated Show resolved Hide resolved
@@ -162,6 +162,13 @@ impl Path {
})
}

/// Parse a url encoded string as a [`Path`], returning a [`Error`] if invalid,
/// as defined on the docstring for [`Path`]
pub fn from_url_path(path: impl AsRef<str>) -> Result<Self, Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of this taking AsRef<Url>?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense, let me do it later today

ps: then we can name it from_url(url: impl AsRef<URL>) instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ps: then we can name it from_url(url: impl AsRef) instead?

I think I would keep it as from_url_path as it will ignore the rest of the URL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

turn out existing code have

pub(crate) fn from_absolute_path_with_base(
path: impl AsRef<std::path::Path>,
base: Option<&Url>,
) -> Result<Self, Error> {
let url = absolute_path_to_url(path)?;
let path = match base {
Some(prefix) => url.path().strip_prefix(prefix.path()).ok_or_else(|| {
Error::PrefixMismatch {
path: url.path().to_string(),
prefix: prefix.to_string(),
}
})?,
None => url.path(),
};

so do you think it makes more sense to use from_url_path(&str)?

d14135d

object_store/src/path/mod.rs Outdated Show resolved Hide resolved
@jychen7 jychen7 requested a review from tustvold February 6, 2023 02:54

/// Parse a url encoded string as a [`Path`], returning a [`Error`] if invalid,
/// as defined on the docstring for [`Path`]
pub fn from_url_path(path: &str) -> Result<Self, Error> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking to define as impl AsRef<str>, but

  1. as_bytes method not found in impl AsRef<str>
  2. NonUnicodeSnafu { path }: the trait From<impl AsRef<str>> is not implemented for String


/// Parse a url encoded string as a [`Path`], returning a [`Error`] if invalid,
/// as defined on the docstring for [`Path`]
pub fn from_url_path(path: &str) -> Result<Self, Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn from_url_path(path: &str) -> Result<Self, Error> {
pub fn from_url_path(path: impl AsRef<str>) -> Result<Self, Error> {
let path = path.as_ref()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

@tustvold
Copy link
Contributor

tustvold commented Feb 6, 2023

I took the liberty of patching this slightly to get it in, thank you 👍

@tustvold tustvold merged commit fb11792 into apache:master Feb 6, 2023
@ursabot
Copy link

ursabot commented Feb 6, 2023

Benchmark runs are scheduled for baseline = 9131c30 and contender = fb11792. fb11792 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@jychen7 jychen7 deleted the 3651-object-store-path-from-url-path branch February 6, 2023 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
object-store Object Store Interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

object_store: support encoded path as input
3 participants