404 - *.html file extension?

[Yttrium-tYcLief]

I've been banging my head against the wall trying to get this set up, and I think I'm nearly there, but now I'm getting error 404's when trying to read back the validation code. You know the error:

Got response code 404, waiting 15 seconds...
Got response code 404, waiting 15 seconds...
Got response code 404, waiting 15 seconds...

Now, in trying to debug this, I think I've figured out what's going on. The trouble is that I don't really know how to fix it in a consistent way. I could override it manually, but it would be a problem for every renewal job in the future.

By checking the Gitlab job status after letsencrypt pushes its commit, I can browse through the job artifacts and see exactly what data it's spitting out. It properly generates the domainname.com/.well-known/acme-challenge/ folder structure, however the file it creates in that folder is called xY6PmHQcg9rp5j2vkdDWS3etCbwFE7MfuBa_zZ8sGqR.html. If I manually try to load that file in Chrome, it shows the response data just fine.

However, letsencrypt isn't looking for xY6PmHQcg9rp5j2vkdDWS3etCbwFE7MfuBa_zZ8sGqR.html, it's looking for xY6PmHQcg9rp5j2vkdDWS3etCbwFE7MfuBa_zZ8sGqR, and receiving a 404.

Is there any way I can either modify my configuration to permit this anomaly, or would it be possible to add a config option for this type of thing upstream? Keep in mind that I have absolutely no experience writing anything in Ruby, so I wouldn't be able to make a PR with the feature, or even fix it myself.

[JustinAiken]

Jekyll gives you a pretty_url option, so that when you do some_link/ with a trailing slash, it'll serve up the URL without needing the .html added at the end. If you have that setting set in jekyll, you can turn on pretty_url in this config to make that work..

Does that work for you, or do you need an option to add the full .html ?

[Yttrium-tYcLief]

I tried that, and maybe something's messed up with my Jekyll config, but browsing the output data doesn't show the expected changed output of xY6PmHQcg9rp5j2vkdDWS3etCbwFE7MfuBa_zZ8sGqR/index.html. It just shows the regular output.

Adding the full *.html would be really helpful.

[JustinAiken]

Created a branch that does that: https://github.com/JustinAiken/jekyll-gitlab-letsencrypt/tree/feature/opt_append_html

In your jekyll app, try it out:

Gemfile

# replace
gem "jekyll-gitlab-letsencrypt"
# with 
gem "jekyll-gitlab-letsencrypt", github: "JustinAiken/jekyll-gitlab-letsencrypt', branch: "opt_append_html"

Jekyll config

gitlab-letsencrypt:
  ... 
  # Add this:
  append_html: true

Then give it a try - if it works for you I'll merge it in and push out a new release

[Yttrium-tYcLief]

Just a heads up, in the last line of your sample config you closed a " with a ' 😉

Also, the branch isn't called "opt_append_html", it's "feature/opt_append_html".

After that though... It's definitely altered its behavior but it still doesn't work. Rather than creating:
.well-known/acme-challenge/lUWBobFSORt5tlEpcplaxJDsVkiiJhBrXxMpKa7oVZo/index.html

It created:
.well-known/acme-challenge/lUWBobFSORt5tlEpcplaxJDsVkiiJhBrXxMpKa7oVZo/.html

So I'm still getting 404 errors.

Thanks for the effort though! I recognize it's a pretty weird and niche problem, and if I knew Ruby it would have been very simple to fix.

[JustinAiken]

Ah, try this... bundle update to pull in the latest commit, then change the config to:

gitlab-letsencrypt:
  ... 
  # Add this:
  append_str: "index.html"

and take out the append_html line

[Yttrium-tYcLief]

Success! Sort of.

The file definitely outputs correctly, and for the first time ever, letsencrypt determines a successful validation. That being said, I... don't know what this error is really pointing to. Googling around doesn't show anything obvious. Any ideas?

I'm going to go ahead and close this issue though, since your new branch has definitely fixed the problem the issue was about.

[JustinAiken]

Timeouts are transitory errors... there's a BadNonce I rescue and retry... maybe should get Timeout too...

Give it another try, see if it fails every time..

[Yttrium-tYcLief]

Yeah, I've been trying every once and awhile for the past two days, just in case it was a load issue on their servers. It times out every single time. There was one time where Gitlab was under such heavy load that it took more than 120 seconds to finish processing, and I got a bad nonce error that time, but right after it said "bad nonce! trying again..", it quit with the same timeout error.

Is 120 seconds too long of a delay for letsencrypt? I would assume not, especially because it's the default setting. I can't really try any delays lower than that though, because that's about how long it takes Gitlab to process my site normally.

[JustinAiken]

It often takes more then 120 seconds for Gitlab to do it's thing - based on your CI job, it may take quite a while (my own blog spends a few minutes compiling nokogiri each time 😛) - 120 is just a nice "let's not start asking until there's a chance it's there"

But in your screenshot, and from what you're described, Gitlab didn't time out - when you see Got response code 200, file is present!, that means it was able to read the challenge file live on your domain, so it's done talking to Gitlab.

The timeout in the screenshot, or if there's a badnonce, is a timeout talking to letsencrypt's API... it's a bit flaky sometimes. I just tried on my own blog, and it worked (120 seconds plus 45 sec of retries)..

One of these days I'll get around to updating to the Acme 2.0 API, which is hopefully a bit more reliable... until then, there's the occasional timeout or other network connection error to acme :/

[Yttrium-tYcLief]

I definitely understood where the issue was - that's why I stated "too long of a delay for letsencrypt", as in the API.

Unfortunately despite trying almost every day, it never seems to communicate properly with letsencrypt.

I'm gunna try later from a different computer, maybe it'll work then. I'll also try another internet connection in case it's the network.

[bear454]

Any chance the "feature/opt_append_html" is going to be merged into master?

[pcercuei]

Same question.

[JustinAiken]

@bear454 @pcercuei - Sorry for the delay..

Just merged that, and pushed out a new release: https://github.com/JustinAiken/jekyll-gitlab-letsencrypt/releases/tag/v0.4.1

Hope it helps!

[bear454]

@bear454 @pcercuei - Sorry for the delay..

Just merged that, and pushed out a new release: https://github.com/JustinAiken/jekyll-gitlab-letsencrypt/releases/tag/v0.4.1

Hope it helps!

I ended up using certbot, and modifying the files in CI:
https://gitlab.com/lfnw/lfnw-blog/blob/master/.gitlab-ci.yml#L31