Long running image builds timeout before completing

[MarkBennett]

I'm developing a large nodejs app and need to run a webpack build on 4,000+ files as part of the building my container image. Unfortunately, this process is so slow that the TaskRun terminates the build at 10 minutes assuming it's died.

This is the output at rio ps:

NAME                     CREATED             ENDPOINT                                          REVISIONS   SCALE     WEIGHT    DETAIL
default/pensive-brown7   About an hour ago   https://pensive-brown7-default.0c6dfy.on-rio.io   v0          0/1       100%      v0: not ready; v0 build failed: TaskRun "default-pensive-brown7-35e37-2bc94" failed to finish within "10m0s"

Given that it's normal in this case for a build to take up to 15 minutes, it would be ideal if there was a way to increase the timeout for my Rio install.

For reference, here's the output of rio info:

Rio Version: v0.4.0 (a8d35ef9)
Rio CLI Version: v0.4.0 (a8d35ef9)
Cluster Domain: 0c6dfy.on-rio.io
Cluster Domain IPs: 159.203.54.33
System Namespace: rio-system

System Components:
Autoscaler status: running
BuildController status: running
CertManager status: running
Grafana status: running
IstioCitadel status: running
IstioPilot status: running
IstioTelemetry status: running
Kiali status: running
Prometheus status: running
Registry status: running
Webhook status: running

Thanks!

[StrongMonkey]

The right fix is to add a field in rio service to expose build-timeout parameter, then pass it to TaskRun.

[MarkBennett]

Thanks for changing this so quickly @daxmc99 and @StrongMonkey! 👍 😄

[daxmc99]

@MarkBennett Could you give this a whirl with rio run --build-timeout 30m <Your Awesome Project>?

[MarkBennett]

Sure! I'll give it a whirl @daxmc99. Is it included in this release?

https://github.com/rancher/rio/releases/tag/v0.5.0-alpha.2

[daxmc99]

Yes it is 👍

[MarkBennett]

Will VERSION_RIO=v0.5.0-alpha.2 curl -sfL https://get.rio.io | sh - work to install the alpha @daxmc99? Just installing it into a temporary container for testing.

[daxmc99]

Try curl -sfL https://get.rio.io | INSTALL_RIO_VERSION=v0.5.0-alpha.2 sh - && rio install to upgrade the server version

[daxmc99]

This can also be verified via k get taskruns.tekton.dev <build of run> -o yaml

[StrongMonkey]

@daxmc99 I assume this is already verified by you, can you confirm and close this issue?

[daxmc99]

@MarkBennett Please comment if you see this is not fixed

[MarkBennett]

I had to rebuild my cluster as my attempt to upgrade v0.4-v.05alpha seemed to really break things. Once that was done, I can confirm that a build that was previously killed after 10 minutes now runs for the length specified in the timeout.

Thanks @daxmc99 and @StrongMonkey! :)