Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gzip compression for "deno compile" #1

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(compile): add gzip compression
  • Loading branch information
DjDeveloperr committed Nov 30, 2020
commit 107a14098ed4cd9e785b9a8f0f7a731b3d1c377e
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/codeStyles/codeStyleConfig.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/contentModel.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/discord.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/encodings.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/indexLayout.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/projectSettingsUpdater.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

104 changes: 104 additions & 0 deletions cli/.idea/.idea.cli.dir/.idea/workspace.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions cli/.idea/.idea.cli.dir/riderModule.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ deno_doc = "0.1.16"
deno_lint = "0.2.11"
deno_web = { path = "../op_crates/web", version = "0.20.0" }
deno_fetch = { path = "../op_crates/fetch", version = "0.12.0" }
flate2 = "1.0"

atty = "0.2.14"
base64 = "0.12.3"
Expand Down
10 changes: 8 additions & 2 deletions cli/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ use std::path::PathBuf;
use std::pin::Pin;
use std::rc::Rc;
use std::sync::Arc;
use flate2::Compression;
use flate2::write::GzEncoder;

fn write_to_stdout_ignore_sigpipe(bytes: &[u8]) -> Result<(), std::io::Error> {
use std::io::ErrorKind;
Expand Down Expand Up @@ -227,13 +229,17 @@ async fn compile_command(
let original_binary_path = std::env::current_exe()?;
let mut original_bin = tokio::fs::read(original_binary_path).await?;

let mut bundle = bundle_str.as_bytes().to_vec();
let mut bundle;

let mut encoder = GzEncoder::new(Vec::new(), Compression::default());
encoder.write_all(&*bundle_str.as_bytes().to_vec()).expect("couldn't write code to binary");
bundle = encoder.finish().expect("invalid compressed code in binary");

let mut magic_trailer = b"DENO".to_vec();
magic_trailer.write_all(&original_bin.len().to_be_bytes())?;

let mut final_bin =
Vec::with_capacity(original_bin.len() + bundle.len() + 12);
Vec::with_capacity(original_bin.len() + bundle.len() + magic_trailer.len());
final_bin.append(&mut original_bin);
final_bin.append(&mut bundle);
final_bin.append(&mut magic_trailer);
Expand Down
49 changes: 22 additions & 27 deletions cli/standalone.rs
Original file line number Diff line number Diff line change
@@ -1,31 +1,31 @@
use crate::colors;
use crate::flags::Flags;
use crate::permissions::Permissions;
use crate::program_state::ProgramState;
use crate::tokio_util;
use crate::worker::MainWorker;
use deno_core::error::AnyError;
use deno_core::futures::FutureExt;
use deno_core::ModuleLoader;
use deno_core::ModuleSpecifier;
use deno_core::OpState;
use std::cell::RefCell;
use std::convert::TryInto;
use std::env::current_exe;
use std::fs::File;
use std::fs::read;
use std::io::Read;
use std::io::Seek;
use std::io::SeekFrom;
use std::pin::Pin;
use std::rc::Rc;

use deno_core::error::AnyError;
use deno_core::futures::FutureExt;
use deno_core::ModuleLoader;
use deno_core::ModuleSpecifier;
use deno_core::OpState;

use crate::colors;
use crate::flags::Flags;
use crate::permissions::Permissions;
use crate::program_state::ProgramState;
use crate::tokio_util;
use crate::worker::MainWorker;
use flate2::read::GzDecoder;

pub fn standalone() {
let current_exe_path =
current_exe().expect("expect current exe path to be known");

let mut current_exe = File::open(current_exe_path)
let mut current_exe = File::open(&current_exe_path)
.expect("expected to be able to open current exe");
let magic_trailer_pos = current_exe
.seek(SeekFrom::End(-12))
Expand All @@ -39,26 +39,21 @@ pub fn standalone() {
let bundle_pos_arr: &[u8; 8] =
bundle_pos.try_into().expect("slice with incorrect length");
let bundle_pos = u64::from_be_bytes(*bundle_pos_arr);
println!(
"standalone bin! bundle starting at {} and ending at {}.",
bundle_pos,
magic_trailer_pos - 1
);
current_exe
.seek(SeekFrom::Start(bundle_pos))
.expect("expected to be able to seek to bundle pos in current exe");

let bundle_len = magic_trailer_pos - bundle_pos;
let mut bundle = String::new();
current_exe
.take(bundle_len)
.read_to_string(&mut bundle)
.expect("expected to be able to read bundle from current exe");
// TODO: check amount of bytes read
let mut compressed = read(&current_exe_path).expect("unable to read exe");

compressed = Vec::from(compressed.split_at(bundle_pos as usize).1);
compressed = Vec::from(compressed.split_at(bundle_len as usize).0);

// println!("standalone bin bundle:\n{}", bundle);
let mut decompress = GzDecoder::new(&compressed as &[u8]);
let mut src = String::new();
decompress.read_to_string(&mut src).unwrap();

let result = tokio_util::run_basic(run(bundle));
let result = tokio_util::run_basic(run(src));
if let Err(err) = result {
eprintln!("{}: {}", colors::red_bold("error"), err.to_string());
std::process::exit(1);
Expand Down
1 change: 0 additions & 1 deletion cli/tests/symlink_to_subdir

This file was deleted.

1 change: 1 addition & 0 deletions cli/tests/symlink_to_subdir
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
subdir/
Empty file modified cli/tests/unit/unit_test_runner.ts
100755 → 100644
Empty file.
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.