Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP R2 API #355

Merged
merged 4 commits into from
Sep 6, 2022
Merged

WIP R2 API #355

merged 4 commits into from
Sep 6, 2022

Conversation

penalosa
Copy link
Contributor

@penalosa penalosa commented Sep 1, 2022

Working (probably), but still writing tests

@penalosa penalosa requested a review from mrbbot September 1, 2022 22:46
@cloudflare-pages
Copy link

cloudflare-pages bot commented Sep 1, 2022

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: a01fd01
Status: ✅  Deploy successful!
Preview URL: https://b2153ba2.cf-miniflare.pages.dev
Branch Preview URL: https://tre-r2.cf-miniflare.pages.dev

View logs

@penalosa penalosa self-assigned this Sep 2, 2022
Copy link
Contributor

@mrbbot mrbbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! 👋 Thanks again for getting to this so quickly. 🙂 I've added a few comments, including some clarification on what I think should be in R2Gateway. Let me know if you have any questions! 🙂

Comment on lines 128 to 130
function throwR2Error(method: Method, status: number, message: string): void {
throw new Error(`R2 ${method} failed: (${status}) ${message}`);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than throwing a generic Error here, I'd throw a subclass of HttpError, similar to KVError here:

export class KVError extends HttpError {}

This will ensure:

  • Correct HTTP response codes get returned
  • Error messages in the runtime match what would occur when deployed

These error messages probably don't need to include the HTTP method too. I think this gets added by the runtime.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed (kinda) in the latest push. I have a subclass of Error export class R2Error extends Error, which encodes the status code & CF error code (I didn't actually think of subclassing HttpError, primarily because each error needs explicitly caught and converted to a response, as the runtime needs each error formatted fairly specifically)

packages/tre/src/plugins/r2/gateway.ts Outdated Show resolved Hide resolved
"R2 put() accepts only nulls, strings, Blobs, ArrayBuffers, ArrayBufferViews, and ReadableStreams as values."
);
}
}

export class R2Gateway {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking with R2Gateway is that it contains shared code between R2Router, and a future R2Bucket class that we might implement (when we start thinking about integration testing for users' workers). R2 bucket bindings are instances of R2Bucket in the runtime.

This means that we probably don't need to return R2Object(Body)s from methods in R2Gateway, as this would be the responsibility of R2Bucket. At this stage, I don't actually think we need the R2Object/R2ObjectBody classes. Similarly, R2Gateway#put should probably just take a Uint8Array value instead of an R2PutValueType. We're always going to pass a Uint8Array from R2Router, and R2PutValueType normalisation should go in R2Bucket.

Essentially, everything that's defined in the runtime's C++ files will go in the future R2Bucket class, and everything that's in the R2 gateway worker should get split between R2Router and R2Gateway classes. 🙂

packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
@penalosa
Copy link
Contributor Author

penalosa commented Sep 5, 2022

Apologies @mrbbot! This wasn't quite ready for review—it should be now. I'll go through and have a check of your comments and fix any that are still relevant with the new code

@penalosa penalosa requested a review from mrbbot September 5, 2022 20:22
Copy link
Contributor

@mrbbot mrbbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! 😃 Thanks again for doing this. I've added a few small comments. 👍

packages/tre/src/plugins/r2/router.ts Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/r2Object.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/r2Object.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/router.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/validator.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/validator.ts Outdated Show resolved Hide resolved
packages/tre/src/plugins/r2/validator.ts Outdated Show resolved Hide resolved
@penalosa penalosa merged commit 5b8f84e into tre Sep 6, 2022
@penalosa penalosa deleted the tre-r2 branch September 6, 2022 12:36
@mrbbot mrbbot added this to the 3.0.0 milestone Sep 9, 2022
@mrbbot mrbbot added the tre Relating to Miniflare 3 label Sep 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tre Relating to Miniflare 3
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants