panic: certificate worker: interface conversion: interface {} is nil, not acme.Authorization

[yoursunny]

What version of the package are you using?

Caddy 2.4.5, which contains certmagic v0.14.5

What are you trying to do?

I have a web server where firewall allows TCP port 443 but blocks TCP port 80.
I'm trying to obtain a certificate from Buypass CA.

What steps did you take?

I inserted the following in Caddyfile:

{
  acme_ca https://api.buypass.com/acme/directory
  email [email protected] # write a real email here
}
https://www.example.net { # write a real hostname here
  handle * {
    respond 404
  }
}

What did you expect to happen, and what actually happened instead?

I'm observing logs in sudo journalctl -fu caddy. I expect either the certificate is obtained via TLS-ALPN-01 challenge, or the certificate cannot be obtained with a proper error message.
Instead, I see the following panic:

Nov 05 04:21:04 ocf0 caddy[6196]: {"level":"info","ts":1636086064.074042,"logger":"tls.obtain","msg":"acquiring lock","identifier":"www.example.net"}
Nov 05 04:21:04 ocf0 caddy[6196]: {"level":"info","ts":1636086064.080959,"logger":"tls.obtain","msg":"lock acquired","identifier":"www.example.net"}
Nov 05 04:21:07 ocf0 caddy[6196]: {"level":"info","ts":1636086067.0777335,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["www.example.net"],"ca":"https://api.buypass.com/acme/directory","account":"[email protected]"}
Nov 05 04:21:07 ocf0 caddy[6196]: {"level":"info","ts":1636086067.0777755,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["www.example.net"],"ca":"https://api.buypass.com/acme/directory","account":"[email protected]"}
Nov 05 04:21:09 ocf0 caddy[6196]: {"level":"info","ts":1636086069.9629297,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"www.example.net","challenge_type":"http-01","ca":"https://api.buypass.com/acme/directory"}
Nov 05 04:21:24 ocf0 caddy[6196]: {"level":"warn","ts":1636086084.9640415,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:21:41 ocf0 caddy[6196]: {"level":"warn","ts":1636086101.7933955,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:21:59 ocf0 caddy[6196]: {"level":"warn","ts":1636086119.2209373,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:22:15 ocf0 caddy[6196]: {"level":"warn","ts":1636086135.8681421,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:22:33 ocf0 caddy[6196]: {"level":"warn","ts":1636086153.046647,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:22:49 ocf0 caddy[6196]: {"level":"warn","ts":1636086169.7631803,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:23:06 ocf0 caddy[6196]: {"level":"warn","ts":1636086186.9451563,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:23:24 ocf0 caddy[6196]: {"level":"warn","ts":1636086204.329125,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:23:41 ocf0 caddy[6196]: {"level":"warn","ts":1636086221.5654945,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:23:58 ocf0 caddy[6196]: {"level":"warn","ts":1636086238.2492294,"logger":"tls.issuance.acme.acme_client","msg":"HTTP request failed; retrying","url":"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1","error":"performing request: Post \"https://api.buypass.com/acme-v02/authz/9ESZ_4mUMcbX-qH0EOwk_VBuaJK4dIpPfJUElrE53XY/1\": net/http: timeout awaiting response headers"}
Nov 05 04:24:00 ocf0 caddy[6196]: {"level":"info","ts":1636086240.687615,"logger":"tls.obtain","msg":"releasing lock","identifier":"www.example.net"}
Nov 05 04:24:00 ocf0 caddy[6196]: 2021/11/05 04:24:00 panic: certificate worker: interface conversion: interface {} is nil, not acme.Authorization
Nov 05 04:24:00 ocf0 caddy[6196]: goroutine 37 [running]:
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*jobManager).worker.func1()
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/async.go:58 +0x65
Nov 05 04:24:00 ocf0 caddy[6196]: panic({0x146c040, 0xc000668a50})
Nov 05 04:24:00 ocf0 caddy[6196]:         runtime/panic.go:1038 +0x215
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/mholt/acmez.(*Client).ObtainCertificateUsingCSR(0xc0002fa468, {0x190d3b8, 0xc000137dd0}, {{0xc0003ea7c8, 0x5}, {0xc0004b87d0, 0x1, 0x1}, 0x1, {0x0, ...}, ...}, ...)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/mholt/[email protected]/client.go:137 +0x1455
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*ACMEManager).doIssue(0xc000137dd0, {0x190d3b8, 0xc000137dd0}, 0xc000226f00, 0x0)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/acmemanager.go:315 +0x19c
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*ACMEManager).Issue(0xc000227200, {0x190d3b8, 0xc000137dd0}, 0x0)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/acmemanager.go:244 +0xa9
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).Issue(0x165e432, {0x190d3b8, 0xc000137dd0}, 0xc0004b8790)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/caddy/[email protected]/modules/caddytls/acmeissuer.go:234 +0xb8
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*Config).obtainCert.func2({0x190d3b8, 0xc000137dd0})
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/config.go:523 +0xa73
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.doWithRetry({0x190d310, 0xc00043ca80}, 0xc0001e11a0, 0xc00072fba8)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/async.go:106 +0x1cc
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*Config).obtainCert(0xc0000bf680, {0x190d310, 0xc00043ca80}, {0xc000045410, 0x12}, 0x0)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/config.go:572 +0x58e
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*Config).ObtainCertAsync(...)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/config.go:427
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*Config).manageOne.func1()
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/config.go:332 +0x6f
Nov 05 04:24:00 ocf0 caddy[6196]: github.com/caddyserver/certmagic.(*jobManager).worker(0x23e0c60)
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/async.go:73 +0x112
Nov 05 04:24:00 ocf0 caddy[6196]: created by github.com/caddyserver/certmagic.(*jobManager).Submit
Nov 05 04:24:00 ocf0 caddy[6196]:         github.com/caddyserver/[email protected]/async.go:50 +0x288

How do you think this should be fixed?

Error conditions such as failure to obtain a certificate should not cause a panic.

[mholt]

Fun, an error path that I didn't anticipate (nor have I seen before). Looks like Buypass' ACME API is really slow. I just need to add the Authorization object to the Problem struct in those cases, or make the type assertion optional.