Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core) deno_core::extension! macro to simplify extension registration #18210

Merged
merged 50 commits into from
Mar 17, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
1e33059
WIP: deno_core::ops!
mmastrac Mar 14, 2023
dd7f871
WIP extension
mmastrac Mar 15, 2023
782225d
WIP extension: napi
mmastrac Mar 15, 2023
ea04949
WIP: deno_websocket extension!
mmastrac Mar 15, 2023
ad224fb
WIP: extension! for deno_webidl and core
mmastrac Mar 15, 2023
910aaa6
Move state into the macro (though it could still live externally)
mmastrac Mar 15, 2023
e543288
Call the state initialization closure using the config fields
mmastrac Mar 15, 2023
f82ff1e
extension! for deno_bench, remove requirement for Default for now
mmastrac Mar 15, 2023
9b3180a
extension! for deno_console and deno_url
mmastrac Mar 15, 2023
0ed5c8c
extension! for deno_cli, deno_test, deno_tsc
mmastrac Mar 15, 2023
c55fda0
extension! for deno_broadcast_channel (required making Config object …
mmastrac Mar 15, 2023
702cfda
extension! for cache, crypto, ffi, webgpu
mmastrac Mar 15, 2023
df0112b
extension! deno_fetch, deno_flash, deno_bench and some internal exten…
mmastrac Mar 16, 2023
c846913
Move ops out of function
mmastrac Mar 16, 2023
81dc32a
Refactor extension! macro slightly
mmastrac Mar 16, 2023
d78cfb8
extension! for cli/build
mmastrac Mar 16, 2023
07044e7
Restore old init_ops/init_ops_and_esm names
mmastrac Mar 16, 2023
2f90862
extension! for webgpu and webgpu_surface
mmastrac Mar 16, 2023
5076f04
ops -> ops_fn
mmastrac Mar 16, 2023
c72730e
extension! deno_fs
mmastrac Mar 16, 2023
4c6f23d
extension! deno_io and deno_http
mmastrac Mar 16, 2023
34150ac
extension! for deno_net, deno_node, deno_load_loading
mmastrac Mar 16, 2023
a563d9e
Update runtime/build and cli/build
mmastrac Mar 16, 2023
37c43fa
extension! deno_tls, deno_web, runtime
mmastrac Mar 16, 2023
fa0eb50
extension! for various deno_* extensions
mmastrac Mar 16, 2023
c7b55b3
Rework extension! to use a struct
mmastrac Mar 16, 2023
0ed4b23
Update tests to use extension! macro
mmastrac Mar 16, 2023
38a0666
Merge branch 'main' into ops_extension
mmastrac Mar 16, 2023
7eb8eb5
Restore extensions after merge conflict
mmastrac Mar 16, 2023
d7bc1e5
Fold ops! into extension!
mmastrac Mar 16, 2023
80b04ca
Trailing comma
mmastrac Mar 16, 2023
4a1ad1d
Fix typo (moved code)
mmastrac Mar 16, 2023
661566d
Add customizer as a 'last resort' and implement extension! for cli
mmastrac Mar 16, 2023
6b39e21
Remove redundant clones
mmastrac Mar 16, 2023
993aa0f
Add deps for deno_fs
mmastrac Mar 16, 2023
6ef38e5
[ci]
mmastrac Mar 16, 2023
bbe8890
Merge branch 'main' into ops_extension
bartlomieju Mar 16, 2023
044c1b7
Add docs for extension!
mmastrac Mar 16, 2023
c16b4ea
Merge branch 'ops_extension' of https://github.com/mmastrac/deno into…
mmastrac Mar 16, 2023
dca0405
One last merge fix
mmastrac Mar 16, 2023
bb7a5fc
Missed deps for deno_websocket
mmastrac Mar 17, 2023
c4fb235
cargo fmt
mmastrac Mar 17, 2023
9983941
Merge branch 'main' into ops_extension
mmastrac Mar 17, 2023
f300f6a
Restore missing comment
mmastrac Mar 17, 2023
9a90368
Slighly cleaner approach for runtime_main extension
mmastrac Mar 17, 2023
3e03787
Construct vector in-place rather than using push
mmastrac Mar 17, 2023
ff419ae
Merge branch 'main' into ops_extension
mmastrac Mar 17, 2023
b009d64
Construct vec! in-place and avoid push
mmastrac Mar 17, 2023
93a9367
Merge branch 'main' of https://github.com/denoland/deno into ops_exte…
mmastrac Mar 17, 2023
0727418
Missed #[inline] on two functions
mmastrac Mar 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Restore old init_ops/init_ops_and_esm names
  • Loading branch information
mmastrac committed Mar 16, 2023
commit 07044e7e32256c837202e90a4aef32751d86d50f
2 changes: 1 addition & 1 deletion bench_util/benches/op_baseline.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ deno_core::ops!(deno_ops, [op_pi_json, op_pi_async, op_nop,]);
deno_core::extension!(bench_setup, ops = deno_ops,);

fn setup() -> Vec<Extension> {
vec![bench_setup::init_runtime()]
vec![bench_setup::init_ops()]
}

#[op]
Expand Down
34 changes: 18 additions & 16 deletions cli/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,11 @@ mod ts {
cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
snapshot_path,
startup_snapshot: None,
extensions: vec![deno_tsc::init_esm_and_state(op_crate_libs, build_libs, path_dts)],
extensions: vec![deno_tsc::init_ops_and_esm(
op_crate_libs,
build_libs,
path_dts,
)],

// NOTE(bartlomieju): Compressing the TSC snapshot in debug build took
// ~45s on M1 MacBook Pro; without compression it took ~1s.
Expand Down Expand Up @@ -314,42 +318,40 @@ mod ts {

fn create_cli_snapshot(snapshot_path: PathBuf) {
let mut extensions: Vec<Extension> = vec![
deno_webidl::deno_webidl::init_runtime(),
deno_console::deno_console::init_runtime(),
deno_url::deno_url::init_runtime(),
deno_webidl::deno_webidl::init_ops(),
deno_console::deno_console::init_ops(),
deno_url::deno_url::init_ops(),
deno_tls::init_ops(),
deno_web::init_ops::<PermissionsContainer>(
deno_web::BlobStore::default(),
Default::default(),
),
deno_fetch::deno_fetch::init_runtime::<PermissionsContainer>(
Default::default(),
),
deno_cache::deno_cache::init_runtime::<SqliteBackedCache>(None),
deno_websocket::deno_websocket::init_runtime::<PermissionsContainer>(
deno_fetch::deno_fetch::init_ops::<PermissionsContainer>(Default::default()),
deno_cache::deno_cache::init_ops::<SqliteBackedCache>(None),
deno_websocket::deno_websocket::init_ops::<PermissionsContainer>(
"".to_owned(),
None,
None,
),
deno_webstorage::deno_webstorage::init_runtime(None),
deno_crypto::deno_crypto::init_runtime(None),
deno_webgpu::deno_webgpu::init_runtime(false),
deno_broadcast_channel::deno_broadcast_channel::init_runtime(
deno_webstorage::deno_webstorage::init_ops(None),
deno_crypto::deno_crypto::init_ops(None),
deno_webgpu::deno_webgpu::init_ops(false),
deno_broadcast_channel::deno_broadcast_channel::init_ops(
deno_broadcast_channel::InMemoryBroadcastChannel::default(),
false, // No --unstable.
),
deno_io::init_ops(Default::default()),
deno_fs::init_ops::<PermissionsContainer>(false),
deno_node::init_ops::<PermissionsContainer>(None), // No --unstable.
deno_node::init_polyfill_ops(),
deno_ffi::deno_ffi::init_runtime::<PermissionsContainer>(false),
deno_ffi::deno_ffi::init_ops::<PermissionsContainer>(false),
deno_net::init_ops::<PermissionsContainer>(
None, false, // No --unstable.
None,
),
deno_napi::deno_napi::init_runtime::<PermissionsContainer>(),
deno_napi::deno_napi::init_ops::<PermissionsContainer>(),
deno_http::init_ops(),
deno_flash::deno_flash::init_runtime::<PermissionsContainer>(false), // No --unstable
deno_flash::deno_flash::init_ops::<PermissionsContainer>(false), // No --unstable
];

let mut esm_files = include_js_files!(
Expand Down
2 changes: 1 addition & 1 deletion cli/lsp/testing/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ async fn test_specifier(
&ps,
specifier.clone(),
PermissionsContainer::new(permissions),
vec![ops::testing::deno_test::init_runtime(
vec![ops::testing::deno_test::init_ops(
sender,
fail_fast_tracker,
filter,
Expand Down
2 changes: 1 addition & 1 deletion cli/lsp/tsc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2818,7 +2818,7 @@ fn op_script_version(
/// server.
fn js_runtime(performance: Arc<Performance>) -> JsRuntime {
JsRuntime::new(RuntimeOptions {
extensions: vec![deno_tsc::init_runtime(performance)],
extensions: vec![deno_tsc::init_ops(performance)],
startup_snapshot: Some(tsc::compiler_snapshot()),
..Default::default()
})
Expand Down
2 changes: 1 addition & 1 deletion cli/ops/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ pub mod bench;
pub mod testing;

pub fn cli_exts(ps: ProcState) -> Vec<Extension> {
vec![deno_cli::init_runtime(ps)]
vec![deno_cli::init_ops(ps)]
}

deno_core::ops!(deno_ops, [op_npm_process_state]);
Expand Down
2 changes: 1 addition & 1 deletion cli/tools/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@ async fn bench_specifier(
&ps,
specifier,
PermissionsContainer::new(permissions),
vec![ops::bench::deno_bench::init_runtime(channel, filter)],
vec![ops::bench::deno_bench::init_ops(channel, filter)],
Default::default(),
)
.await?;
Expand Down
2 changes: 1 addition & 1 deletion cli/tools/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -723,7 +723,7 @@ async fn test_specifier(
ps,
specifier,
PermissionsContainer::new(permissions),
vec![ops::testing::deno_test::init_runtime(
vec![ops::testing::deno_test::init_ops(
sender,
fail_fast_tracker,
options.filter,
Expand Down
4 changes: 2 additions & 2 deletions cli/tsc/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ fn get_asset_texts_from_new_runtime() -> Result<Vec<AssetText>, AnyError> {
// the assets are stored within the typescript isolate, so take them out of there
let mut runtime = JsRuntime::new(RuntimeOptions {
startup_snapshot: Some(compiler_snapshot()),
extensions: vec![deno_cli_tsc::init_runtime()],
extensions: vec![deno_cli_tsc::init_ops()],
..Default::default()
});
let global =
Expand Down Expand Up @@ -861,7 +861,7 @@ pub fn exec(request: Request) -> Result<Response, AnyError> {

let mut runtime = JsRuntime::new(RuntimeOptions {
startup_snapshot: Some(compiler_snapshot()),
extensions: vec![deno_cli_tsc::init_runtime(
extensions: vec![deno_cli_tsc::init_ops(
Rc::new(request),
root_map,
remapped_specifiers,
Expand Down
6 changes: 3 additions & 3 deletions core/extensions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ macro_rules! extension {
}

#[allow(dead_code)]
pub fn init_esm $( < $( $param : $type + Clone + 'static ),+ > )? () -> $crate::Extension {
pub fn init_js_only $( < $( $param : $type + Clone + 'static ),+ > )? () -> $crate::Extension {
let mut ext = ext();
// If esm or JS was specified, add JS files
with_js(&mut ext);
Expand All @@ -241,7 +241,7 @@ macro_rules! extension {
}

#[allow(dead_code)]
pub fn init_esm_and_state $( < $( $param : $type + Clone + 'static ),+ > )? ( $( $( $config_id : $config_type ),* )? ) -> $crate::Extension {
pub fn init_ops_and_esm $( < $( $param : $type + Clone + 'static ),+ > )? ( $( $( $config_id : $config_type ),* )? ) -> $crate::Extension {
let mut ext = ext();
// If esm or JS was specified, add JS files
with_js(&mut ext);
Expand All @@ -251,7 +251,7 @@ macro_rules! extension {
}

#[allow(dead_code)]
pub fn init_runtime $( < $( $param : $type + Clone + 'static ),+ > )? ( $( $( $config_id : $config_type ),* )? ) -> $crate::Extension {
pub fn init_ops $( < $( $param : $type + Clone + 'static ),+ > )? ( $( $( $config_id : $config_type ),* )? ) -> $crate::Extension {
mmastrac marked this conversation as resolved.
Show resolved Hide resolved
let mut ext = ext();
with_ops $( ::<($( $param ),+)> )?(&mut ext);
with_state_and_middleware $( ::<($( $param ),+)> )?(&mut ext, $( $( $config_id , )* )? );
Expand Down
4 changes: 2 additions & 2 deletions core/runtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -323,11 +323,11 @@ impl JsRuntime {
if !has_startup_snapshot {
options
.extensions
.insert(0, crate::ops_builtin::core::init_esm());
.insert(0, crate::ops_builtin::core::init_ops_and_esm());
} else {
options
.extensions
.insert(0, crate::ops_builtin::core::init_runtime());
.insert(0, crate::ops_builtin::core::init_ops());
}

let ops = Self::collect_ops(&mut options.extensions);
Expand Down
4 changes: 2 additions & 2 deletions ext/url/benches/url_ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ use deno_core::ExtensionFileSourceCode;

fn setup() -> Vec<Extension> {
vec![
deno_webidl::deno_webidl::init_esm(),
deno_url::deno_url::init_esm(),
deno_webidl::deno_webidl::init_ops_and_esm(),
deno_url::deno_url::init_ops_and_esm(),
Extension::builder("bench_setup")
.esm(vec![ExtensionFileSource {
specifier: "ext:setup".to_string(),
Expand Down
6 changes: 3 additions & 3 deletions ext/web/benches/encoding.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ impl deno_web::TimersPermission for Permissions {

fn setup() -> Vec<Extension> {
vec![
deno_webidl::deno_webidl::init_esm(),
deno_url::deno_url::init_esm(),
deno_console::deno_console::init_esm(),
deno_webidl::deno_webidl::init_ops_and_esm(),
deno_url::deno_url::init_ops_and_esm(),
deno_console::deno_console::init_ops_and_esm(),
deno_web::init_ops_and_esm::<Permissions>(BlobStore::default(), None),
Extension::builder("bench_setup")
.esm(vec![ExtensionFileSource {
Expand Down
6 changes: 3 additions & 3 deletions ext/web/benches/timers_ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ impl deno_web::TimersPermission for Permissions {

fn setup() -> Vec<Extension> {
vec![
deno_webidl::deno_webidl::init_esm(),
deno_url::deno_url::init_esm(),
deno_console::deno_console::init_esm(),
deno_webidl::deno_webidl::init_ops_and_esm(),
deno_url::deno_url::init_ops_and_esm(),
deno_console::deno_console::init_ops_and_esm(),
deno_web::init_ops_and_esm::<Permissions>(BlobStore::default(), None),
Extension::builder("bench_setup")
.esm(vec![
Expand Down
46 changes: 37 additions & 9 deletions ext/webgpu/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
#![warn(unsafe_op_in_unsafe_fn)]

use deno_core::error::AnyError;
use deno_core::include_js_files;
use deno_core::op;
use deno_core::Extension;
use deno_core::ExtensionBuilder;
use deno_core::OpState;
use deno_core::Resource;
use deno_core::ResourceId;
Expand Down Expand Up @@ -114,20 +117,45 @@ impl Resource for WebGpuQuerySet {
}
}

deno_core::extension!(deno_webgpu,
deps = [ deno_webidl, deno_web ],
esm = [ "01_webgpu.js", "02_idl_types.js" ],
config = {
unstable: bool,
},
state = |state, unstable| {
// TODO: The WebGPU extension fails with the extension! macro
pub mod deno_webgpu {
use super::*;

pub fn init_ops_and_esm(unstable: bool) -> Extension {
super::init_ops_and_esm(unstable)
}

pub fn init_ops(unstable: bool) -> Extension {
super::init_ops(unstable)
}
}

fn ext() -> ExtensionBuilder {
Extension::builder_with_deps(
env!("CARGO_PKG_NAME"),
&["deno_webidl", "deno_web"],
)
}

fn ops(ext: &mut ExtensionBuilder, unstable: bool) -> &mut ExtensionBuilder {
ext.ops(deno_ops()).state(move |state| {
// TODO: check & possibly streamline this
// Unstable might be able to be OpMiddleware
// let unstable_checker = state.borrow::<super::UnstableChecker>();
// let unstable = unstable_checker.unstable;
state.put(Unstable(unstable));
},
);
})
}

pub fn init_ops_and_esm(unstable: bool) -> Extension {
ops(&mut ext(), unstable)
.esm(include_js_files!("01_webgpu.js", "02_idl_types.js",))
.build()
}

pub fn init_ops(unstable: bool) -> Extension {
ops(&mut ext(), unstable).build()
}

fn deserialize_features(features: &wgpu_types::Features) -> Vec<&'static str> {
let mut return_features: Vec<&'static str> = vec![];
Expand Down
2 changes: 1 addition & 1 deletion ext/webidl/benches/dict.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use deno_core::ExtensionFileSourceCode;

fn setup() -> Vec<Extension> {
vec![
deno_webidl::deno_webidl::init_esm(),
deno_webidl::deno_webidl::init_ops_and_esm(),
Extension::builder("deno_webidl_bench")
.esm(vec![ExtensionFileSource {
specifier: "ext:setup".to_string(),
Expand Down
37 changes: 23 additions & 14 deletions runtime/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -253,33 +253,42 @@ mod startup_snapshot {
.build();

let mut extensions: Vec<Extension> = vec![
deno_webidl::deno_webidl::init_esm(),
deno_console::deno_console::init_esm(),
deno_url::deno_url::init_esm(),
deno_webidl::deno_webidl::init_ops_and_esm(),
deno_console::deno_console::init_ops_and_esm(),
deno_url::deno_url::init_ops_and_esm(),
deno_tls::init_ops(),
deno_web::init_ops_and_esm::<Permissions>(
deno_web::BlobStore::default(),
Default::default(),
),
deno_fetch::deno_fetch::init_esm::<Permissions>(),
deno_cache::deno_cache::init_esm::<SqliteBackedCache>(),
deno_websocket::deno_websocket::init_esm::<Permissions>(),
deno_webstorage::deno_webstorage::init_esm(),
deno_crypto::deno_crypto::init_esm(),
deno_webgpu::deno_webgpu::init_esm(),
deno_broadcast_channel::deno_broadcast_channel::init_esm::<
deno_fetch::deno_fetch::init_ops_and_esm::<Permissions>(
Default::default(),
),
deno_cache::deno_cache::init_ops_and_esm::<SqliteBackedCache>(None),
deno_websocket::deno_websocket::init_ops_and_esm::<Permissions>(
"".to_owned(),
None,
None,
),
deno_webstorage::deno_webstorage::init_ops_and_esm(None),
deno_crypto::deno_crypto::init_ops_and_esm(None),
deno_webgpu::deno_webgpu::init_ops_and_esm(false),
deno_broadcast_channel::deno_broadcast_channel::init_ops_and_esm::<
InMemoryBroadcastChannel,
>(),
deno_ffi::deno_ffi::init_esm::<Permissions>(),
>(
deno_broadcast_channel::InMemoryBroadcastChannel::default(),
false,
), // No --unstable.
deno_ffi::deno_ffi::init_ops_and_esm::<Permissions>(false),
deno_net::init_ops_and_esm::<Permissions>(
None, false, // No --unstable.
None,
),
deno_napi::deno_napi::init_esm::<Permissions>(),
deno_napi::deno_napi::init_ops_and_esm::<Permissions>(),
deno_http::init_ops_and_esm(),
deno_io::init_ops_and_esm(Default::default()),
deno_fs::init_ops_and_esm::<Permissions>(false),
deno_flash::deno_flash::init_esm::<Permissions>(), // No --unstable
deno_flash::deno_flash::init_ops_and_esm::<Permissions>(false), // No --unstable
runtime_extension,
// FIXME(bartlomieju): these extensions are specified last, because they
// depend on `runtime`, even though it should be other way around
Expand Down
Loading