Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Npm: download the required binaries during installation #188

Merged
merged 17 commits into from
Jun 2, 2021
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ dist/
.rubygems/pkg/
.rubygems/libexec/
.npm/bin/
!.npm/bin/*.js
package.json
!.npm/package.json
node_modules/
yarn.lock
package-lock.json
7 changes: 4 additions & 3 deletions .npm/bin/index.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
#!/usr/bin/env node

var spawn = require('child_process').spawn;
const { getExePath } = require('../get-exe');
const path = require("path")
const extension = ["win32", "cygwin"].includes(process.platform) ? ".exe" : ""
const exePath = path.join(__dirname, `lefthook${extension}`)

var command_args = process.argv.slice(2);

var child = spawn(
getExePath(),
exePath,
command_args,
{ stdio: "inherit" });

Expand Down
36 changes: 0 additions & 36 deletions .npm/get-exe.js

This file was deleted.

3 changes: 3 additions & 0 deletions .npm/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,8 @@
],
"scripts": {
"postinstall": "node postinstall.js"
},
"dependencies": {
"node-downloader-helper": "^1.0.18"
}
}
80 changes: 74 additions & 6 deletions .npm/postinstall.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,78 @@
if (!process.env.CI) {
const { spawnSync } = require('child_process');
const { getExePath } = require('./get-exe');
const { spawnSync } = require("child_process")

const iswin = ["win32", "cygwin"].includes(process.platform)

async function install() {
if (process.env.CI) {
return
}
const exePath = await downloadBinary()
if (!iswin) {
const { chmodSync } = require("fs")
chmodSync(exePath, "755")
}
// run install
spawnSync(getExePath(), ['install', '-f'], {
spawnSync(exePath, ["install", "-f"], {
cwd: process.env.INIT_CWD || process.cwd,
stdio: 'inherit',
});
stdio: "inherit",
})
}

function getDownloadURL() {
// Detect OS
// https://nodejs.org/api/process.html#process_process_platform
let goOS = process.platform
let extension = ""
if (iswin) {
goOS = "windows"
extension = ".exe"
}

// Convert the goOS to the os name in the download URL
let downloadOS = goOS === "darwin" ? "macOS" : goOS
downloadOS = `${downloadOS.charAt(0).toUpperCase()}${downloadOS.slice(1)}`

// Detect architecture
// https://nodejs.org/api/process.html#process_process_arch
let arch = process.arch
switch (process.arch) {
case "x64": {
arch = "x86_64"
break
}
case "x32":
case "ia32": {
arch = "i386"
break
}
}
const version = require("./package.json").version

return `https://github.com/evilmartians/lefthook/releases/download/v${version}/lefthook_${version}_${downloadOS}_${arch}${extension}`
}

const { DownloaderHelper } = require("node-downloader-helper")
const path = require("path")

async function downloadBinary() {
// TODO zip the binaries to reduce the download size
aminya marked this conversation as resolved.
Show resolved Hide resolved
const downloadURL = getDownloadURL()
const extension = iswin ? ".exe" : ""
const fileName = `lefthook${extension}`
const binDir = path.join(__dirname, "bin")
const dl = new DownloaderHelper(downloadURL, binDir, {
fileName,
retry: { maxRetries: 5, delay: 50 },
})
dl.on("error", (e) => {
e.message = `Failed to download ${fileName}: ${e.message} while fetching ${downloadURL}`
throw e
Copy link
Contributor Author

@aminya aminya May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does throwing the error here prevent retrying? What I had here allowed the code to retry downloading only throw the error in the end.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this is good question.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it actually prevent retrying. But I believe it is a bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should revert this change. What I had here allowed retrying, and also printed the actual error message.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the problem is that printing doesn't work (at least with yarn). I will experiment with error throwing again later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could replace this throw e with console.error(e), and then revert what I had to throw the last error.

Suggested change
throw e
console.error(e)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

console.error output isn't visible when yarn is used to install package.

Copy link
Contributor Author

@aminya aminya May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then there is no way to print the intermediate error messages with Yarn. I recommend throwing the last error for those on Yarn and let the algorithm work normally for others. This is not a bug in node-downloader-helper. The error callback expects a working code, and if we again rethrow the error in the error callback, we are doing redundant work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted my commit

I couldn't make yarn to print error message in a descriptive way I want because yarn is exiting on the first error event thrown (and I insist that this is caused by the bug in node-download helper, see hgouveia/node-downloader-helper#57 (comment)).

I give up. Let it be so.

})
await dl.start()
return path.join(binDir, fileName)
aminya marked this conversation as resolved.
Show resolved Hide resolved
}

// start:
install().catch((e) => {
throw e
})
12 changes: 11 additions & 1 deletion cmd/templates.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package cmd
import (
"bytes"
"embed"
"runtime"
"text/template"

"github.com/spf13/afero"
Expand All @@ -14,6 +15,7 @@ var templatesFS embed.FS
type hookTmplData struct {
AutoInstall string
HookName string
Extension string
}

func hookTemplate(hookName string, fs afero.Fs) []byte {
Expand All @@ -22,6 +24,7 @@ func hookTemplate(hookName string, fs afero.Fs) []byte {
err := t.Execute(buf, hookTmplData{
AutoInstall: autoInstall(hookName, fs),
HookName: hookName,
Extension: getExtension(),
})
check(err)

Expand All @@ -40,5 +43,12 @@ func autoInstall(hookName string, fs afero.Fs) string {
return ""
}

return "# lefthook_version: " + configChecksum(fs) + "\n\ncall_lefthook \"lefthook install\""
return "# lefthook_version: " + configChecksum(fs) + "\n\ncall_lefthook \"install\""
}

func getExtension() string {
if runtime.GOOS == "windows" {
return ".exe"
}
return ""
}
14 changes: 7 additions & 7 deletions cmd/templates/hook.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ call_lefthook()
{
if lefthook -h >/dev/null 2>&1
then
eval $1
elif test -f "$dir/node_modules/@arkweid/lefthook/bin/lefthook"
eval lefthook $1
aminya marked this conversation as resolved.
Show resolved Hide resolved
elif test -f "$dir/node_modules/@arkweid/lefthook/bin/lefthook{{.Extension}}"
then
eval $dir/node_modules/@arkweid/lefthook/bin/$1
eval "$dir/node_modules/@arkweid/lefthook/bin/lefthook{{.Extension}} $1"
elif bundle exec lefthook -h >/dev/null 2>&1
then
bundle exec $1
bundle exec lefthook $1
elif npx lefthook -h >/dev/null 2>&1
then
npx $1
npx lefthook $1
elif yarn lefthook -h >/dev/null 2>&1
then
yarn $1
yarn lefthook $1
else
echo "Can't find lefthook in PATH"
fi
}

{{.AutoInstall}}

call_lefthook "lefthook run {{.HookName}} $@"
call_lefthook "run {{.HookName}} $@"
12 changes: 6 additions & 6 deletions spec/fixtures/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ call_lefthook()
{
if lefthook -h >/dev/null 2>&1
then
eval $1
eval lefthook $1
elif test -f "$dir/node_modules/@arkweid/lefthook/bin/lefthook"
then
eval $dir/node_modules/@arkweid/lefthook/bin/$1
eval "$dir/node_modules/@arkweid/lefthook/bin/lefthook $1"
elif bundle exec lefthook -h >/dev/null 2>&1
then
bundle exec $1
bundle exec lefthook $1
elif npx lefthook -h >/dev/null 2>&1
then
npx $1
npx lefthook $1
elif yarn lefthook -h >/dev/null 2>&1
then
yarn $1
yarn lefthook $1
else
echo "Can't find lefthook in PATH"
fi
}



call_lefthook "lefthook run pre-commit $@"
call_lefthook "run pre-commit $@"
12 changes: 6 additions & 6 deletions spec/fixtures/pre-push
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ call_lefthook()
{
if lefthook -h >/dev/null 2>&1
then
eval $1
eval lefthook $1
elif test -f "$dir/node_modules/@arkweid/lefthook/bin/lefthook"
then
eval $dir/node_modules/@arkweid/lefthook/bin/$1
eval "$dir/node_modules/@arkweid/lefthook/bin/lefthook $1"
elif bundle exec lefthook -h >/dev/null 2>&1
then
bundle exec $1
bundle exec lefthook $1
elif npx lefthook -h >/dev/null 2>&1
then
npx $1
npx lefthook $1
elif yarn lefthook -h >/dev/null 2>&1
then
yarn $1
yarn lefthook $1
else
echo "Can't find lefthook in PATH"
fi
}



call_lefthook "lefthook run pre-push $@"
call_lefthook "run pre-push $@"