Plugin crashing docker build with node-alpine image

[prateekrastogi]

While trying to create a docker image taking node-alpine 8.11.0 as base image, the install steps of the deps of this plugin are crashing the build process. Here is relevant portion of build log:

[email protected] postinstall /frontend/node_modules/cwebp-bin

node lib/install.js

⚠ spawn /frontend/node_modules/cwebp-bin/vendor/cwebp ENOENT
⚠ cwebp pre-build test failed
ℹ compiling from source
✖ Error: ./configure --disable-shared --prefix="/frontend/node_modules/cwebp-bin/vendor" --bindir="/frontend/node_modules/cwebp-bin/vendor" && make && make install
Command failed: ./configure --disable-shared --prefix="/frontend/node_modules/cwebp-bin/vendor" --bindir="/frontend/node_modules/cwebp-bin/vendor"
configure: error: in /frontend/node_modules/cwebp-bin/1781a0ab-70da-4226-aea4-05978656b745': configure: error: no acceptable C compiler found in $PATH See config.log' for more details

at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:567:12)

[email protected] postinstall /frontend/node_modules/gifsicle
node lib/install.js

⚠ spawn /frontend/node_modules/gifsicle/vendor/gifsicle ENOENT
⚠ gifsicle pre-build test failed
ℹ compiling from source
✖ Error: autoreconf -ivf && ./configure --disable-gifview --disable-gifdiff --prefix="/frontend/node_modules/gifsicle/vendor" --bindir="/frontend/node_modules/gifsicle/vendor" &&
make install
Command failed: autoreconf -ivf
/bin/sh: autoreconf: not found

at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:567:12)

[email protected] postinstall /frontend/node_modules/mozjpeg
node lib/install.js

⚠ spawn /frontend/node_modules/mozjpeg/vendor/cjpeg ENOENT
⚠ mozjpeg pre-build test failed
ℹ compiling from source
✖ Error: autoreconf -fiv && ./configure --disable-shared --disable-dependency-tracking --with-jpeg8 --prefix="/frontend/node_modules/mozjpeg/vendor" --bindir="/frontend/node_modules/mozjpeg/vendor" --libdir="/frontend/node_modules/mozjpeg/vendor" && make -j2 && make install -j2
Command failed: autoreconf -fiv
/bin/sh: autoreconf: not found

at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:567:12)

[email protected] postinstall /frontend/node_modules/optipng-bin
node lib/install.js

⚠ spawn /frontend/node_modules/optipng-bin/vendor/optipng ENOENT
⚠ optipng pre-build test failed
ℹ compiling from source
✖ Error: ./configure --with-system-zlib --prefix="/frontend/node_modules/optipng-bin/vendor" --bindir="/frontend/node_modules/optipng-bin/vendor" && make install
Command failed: ./configure --with-system-zlib --prefix="/frontend/node_modules/optipng-bin/vendor" --bindir="/frontend/node_modules/optipng-bin/vendor"

at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)

[email protected] postinstall /frontend/node_modules/pngquant-bin
node lib/install.js

⚠ spawn /frontend/node_modules/pngquant-bin/vendor/pngquant ENOENT
⚠ pngquant pre-build test failed
ℹ compiling from source
✔ pngquant pre-build test passed successfully
✖ Error: pngquant failed to build, make sure that libpng-dev is installed
at Promise.all.then.arr (/frontend/node_modules/pngquant-bin/node_modules/bin-build/node_modules/execa/index.js:231:11)
at
at process._tickCallback (internal/process/next_tick.js:188:7)
npm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: node lib/install.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-04-02T11_13_47_451Z-debug.log

Are there any clean ways to skip pre and post build steps of those deps in the plugin itself to avoid any issues in containerized environment?

[unregistered]

I had the same issue. But I was able to solve it by making next-optimized-images a devDependency and not including it in next.config.js when running in production.

[prateekrastogi]

@unregistered Are you then running build in dev environment, and then, copy the build folder in docker image instead of building the next.js project inside the docker build?

[unregistered]

@prateekrastogi yes exactly that.

[prateekrastogi]

@unregistered But, this might not be always feasible in large ci enabled team projects with many stakeholders. That's why I was searching for a cleaner workaround.

[cyrilwanner]

I've only used the node:8.11 docker image with this plugin before but I tried to run my project on node:alpine and got the same issues as you have described.

The solution @unregistered posted may solve the problem temporarily, but I agree with you that there should be a cleaner solution, so I started to debug it and tried to find a solution.

First of all, image optimization is quite a heavy task, this is why imagemin (the underlying library) is using compiled binaries for it. But they are obviously not compiled for the alpine docker images and some libraries required for building them are missing because the goal of the alpine images is to have the smallest size possible.

---

So one possibility would be to add the missing packages/libraries required for building the binaries.
The following Dockerfile worked for me:

FROM node:alpine

RUN apk add --no-cache \
    autoconf \
    automake \
    bash \
    g++ \
    libc6-compat \
    libjpeg-turbo-dev \
    libpng-dev \
    make \
    nasm

# build your next.js app now..

You should now be able to create the build inside your docker container but the docker image now also got bigger (in my case from 68MB [node:alpine] to 259MB) but it is still smaller than the node:8.11 image (673MB). You could remove all packages except libc6-compat, libpng-dev and libjpeg-turbo-dev after your build which should make the final image size smaller again. But it also ends up in longer build times because the imagemin libraries need to get rebuilt from the source every time.

---

Another solution would be to once build the binaries for the alpine docker images and then copy them to your docker image before building the next.js app. For this, only the three packages from the previous paragraph would be required and you won't need to rebuild all binaries on every CI run. We would just have to find the correct time to copy the binaries into the image (after npm installed the dependencies and before the postinstall scripts get executed). I already have them built for the alpine image and have the process already in my mind, so if you want, I can try to create a minimalistic docker image already containing the built binaries for the alpine image which you then can extend instead of the normal alpine one. The size of the docker image will probably be around 71MB instead of 68MB.

Would one of these two solutions work for you? Or did you come up with another one in the meantime?

[prateekrastogi]

@cyrilwanner Thanks for listing detailed approaches. I decided to build and remove build time system dependencies as the branch from which docker image is build in my workflow is generally not that frequently updated. Also, using pre-built binaries will create an additional upstream dependency which itself needs to be updated for multiple node releases. Although, docker intermediate image immutability made that build-install-remove portion of Dockerfile slightly dense.

[genox]

I just hit the same issue. Building dependencies from source during CI/CD to be able to run what is essentially a prebuild task, the increased build time, image size and build complexity makes it really hard to justify using this plugin in production compared to running a binary installable imageoptimizer as a pre-build task like svgo or imageoptim on your asset src directory.

Not to belittle your work, just to provide feedback. Maybe you can reconsider some features to allow it to be more widely used. Less is more. Maybe. :)

[cyrilwanner]

Hi @genox
Thank you for your response and opinion on this topic!
I have noticed this too but didn't have much option as this whole plugin is based on another package (img-loader) for optimizing the images. But they recently released a major update with which it should be possible to make everything optional, even the dependencies and used libraries for optimizing the images. So everyone can install/require the dependencies in the step/environment which suits them. I'm planning a complete rewrite of this plugin very soon which takes advantage of this and hopefully resolves many issues which we currently have.

[genox]

I understand the situation. I like what this module does and will definately revisit once those issues have been resolved. Greetings from Solothurn .. ;)

[yoiang]

Has anyone figured out a solution that would work with Vercel/Zeit besides the proposed building elsewhere? Cheers!

[prateekrastogi]

@yoiang Zeit has disabled docker deploys long ago and so did other up-coming providers. Try to argue for open docker in "binder" ;-)

[cyrilwanner]

Hello, I'm actively working on the next major version of this plugin which no longer depends on native binaries to optimize images but uses WebAssembly instead, so the build can run everywhere and it also works with Vercel/Zeit.

It is still a canary version since not all currently existing features are implemented (but almost, only GIF optimization, ?trace and ?sprite are missing). So if you don't depend on them, I suggest to try the canary version and read my comment here.

[prateekrastogi]

WebAssembly hardly have any performance benefits over optimized javascript runtime. Overall, WebAssembly as whole is a dustbin effort.

[yoiang]

@cyrilwanner whoo! that's exciting, I'll check it out, thank you!

[prateekrastogi]

@cyrilwanner The recent discovery of "Quantum Computing via memory address collapse" might be the blame for performance discrepancies.

[cyrilwanner]

WebAssembly hardly have any performance benefits over optimized javascript runtime. Overall, WebAssembly as whole is a dustbin effort.

I know that, and I didn't choose it because of performance reasons. But it allows us to easily run C/Rust/... code in a Node environment without any native binaries or other requirements. And almost all good image optimization libraries are written in something else than javascript. So without WebAssembly, the exact problem of this issue (image optimization requires native binaries) would not be solvable, unless you want to implement your own image optimization library in node.. It is up to you if you want to call it a dustbin effort in general, but for this use-case, I think it made perfect sense.

[prateekrastogi]

Rust web frameworks suffer same kind of performance issues. I believe FFI to node via rust/WebAssembly will invoke entire I/O stack in node, thereby, performance penalty will be in order of magnitude 10. See if that compares with "Space-Time Complexity" of native node options without requiring re-write.