This is forked from letsencrypt-fromscratch
This is a (pretty detailed) how-to on building a simple ACME client from scratch, able to issue real certificates from Let's Encrypt. I've skipped things like error handling, object orientedness, tests - but not much tweaking would be needed for the client to be production-ready.
The code for the finished client is in client.rb
.
This guide assumes no particular knowledge of TLS/SSL, cryptography or ACME - a general understanding of programming, HTTP and REST APIs is probably needed. It would also be useful to have a vague idea of what Public-key cryptography is.
Hopefully this guide is useful to anyone looking to build a Let's Encrypt client, or anyone looking to understand more about how LE/ACME works. Following the guide, you should be able to create a fully fledged LE client and issue a valid certificate in less than an hour. The guide does assume you control a domain name.
Our specimen site is a static website powered by nginx, using DNSimple as the DNS provider (see Appendix 3: Our example site setup). The mechanics of how we pass LE's challenges are based on this sample setup - but treat these just as illustrative examples.
The guide and client code are all MIT licensed.
This example code is written in Ruby (I used 2.3), and is largely dependency free (apart from OpenSSL). We use HTTParty and Nitlink for more convenient API requests - but you could use vanilla Net::HTTP
if you're a masochist π. And we'll use additional gems to upload files and provision DNS records.
The choice of language is meant to be a background factor - the guide is (hopefully) illustrative & understandable even if you're not familiar with/interested in Ruby.
I heavily referenced Daniel Roesler's absolutely awesome acme-tiny and the ACME spec while writing this tutorial. I'd recommend checking out both as a supplement to this guide. Image credits at the bottom.
- Loading our client key-pair
- Constructing a Let's Encrypt API request
- Registering with Let's Encrypt
- Passing the challenge
- Issuing the certificate π
- Appendix 1: Installing and testing the certificate
- Appendix 2: The trust chain & intermediate certificates
- Appendix 3: Our example site setup
- Appendix 4: Multiple subdomains
- Appendix 5: Key size
- Appendix 6: IDN support
- Appendix 7: Using EC client keys
- Appendix 8: Certificate expiry and revocation
- Further reading
- Image credits
- Author
- Changelog
The process of generating our certificate heavily depends on have a client key - or, more accurately key-pair (comprising our public key and private key).
We'll share our public key with Let's Encrypt when we register, and sign all our requests with our private key - Let's Encrypt can use our public key to ensure our requests are genuinely from us (that they've been signed by our private key). We'll never share our private key with Let's Encrypt. We won't share it with any 3rd parties; although our web-server (nginx in our example app) will need access to it in order to encrypt the data it sends to clients.
It's fine to use existing SSH keys, if you've already got them generated and they're long enough:
openssl rsa -in ~/.ssh/id_rsa -text -noout | head -n 1
If you see Private-Key: (2048 bit)
or Private-Key: (4096 bit)
you're good to go (if you're interested, there's more info about key size in Appendix 5). Otherwise, we'll need to generate them - Github has great instructions on how. Let's begin by loading our key-pair into Ruby:
require 'openssl'
client_key_path = File.expand_path('~/.ssh/id_rsa')
client_key = OpenSSL::PKey::RSA.new IO.read(client_key_path)
If our key is encrypted with a passphrase, we'll need to provide that as a 2nd argument:
client_key = OpenSSL::PKey::RSA.new(IO.read(client_key_path), 'letmein')
The first, and probably hardest step, is constructing requests in the very particular format that Let's Encrypt demands. It's important to remember though, that in principle, the Let's Encrypt API is the same as any other API.
For example, using the Github API I can programatically create an issue, by making a POST
request to the target repo's /issues
endpoint with a JSON payload that includes the issue title and body:
POST https://api.github.com/repos/alexpeattie/letsencrypt-fromscratch/issues
{
"title": "Bad examples",
"body": "The code examples in the guide are hard to understand!"
}
The key difference with the Let's Encrypt API is we can't just send our JSON payload in a nice human-readable format as above, because we'll be signing it with our client private key to prove our identity. This is what a request to the Let's Encrypt API looks like:
POST https://acme-v01.api.letsencrypt.org/acme/new-cert
{
"payload": "eyJyZXNvdXJjZSI6Im5ldy1jZXJ0IiwiY3NyIjoiTUlJRVhEQ0NBa1FDQVFBd0Z6RVZNQk1HQTFVRUF3d01abWxzWlhNdWNHVm5MbU52TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUE2ZG9JNWdlc1VWZVV2czJXN1h3LV9JcDg2eFl3ZnV0MDVNWE1aYWpWa3lMS1lhNHpjdGs3Y2hIN1ZuQWsxVF9uTXNaM0hYTlQ3X0J0R1hkYnlJR0FqRXhpR3F4cm5LejJqSS1JTVRNU1RKSklmRVhDUVJqUkx2U0c2S3VYbXk2aGhkS3BLMkpRam10OTh0QmxUY0NxbFFKNGRZWV9oMVFCTmYwZmUwN3p4T24zUXlaeU9Da05GMkdGQmZoSWZqTGRuVXJCbDBSejlTSUhLZkZTWW13SldKMTBBLWJiNVdRM2FkUWlNWF83amhYWHVBdUdDZnRBZ2h1UGdPWjlTalJXYVBpalNkOUxERWk1Y2pCalFsN1o4a0ZKTnV0VndSQlNFTDFIQVVNWE9ndkxKLW5mVjV4Tm15VHdmYTRsdXV4WEtsVnpJZFlmZDRUZWV1NHhwUTAxb29vQ0dLRUVCZ3VMQzdQLUtjemg4MUxXaTZtcExIRVZwOTNzWi1QZDZvNlROMFlabVZjaUwtNlJpTGRXY2hUeEtkbjNvTS1UYmRBTUVxb3VmTU5JYkh6LUVHREFxUkhGOUxCTU43bFlPcWJ0dWFmcjduN1EtVmQxN19KTGIxcnpONVFmclZvd2o4cUJpUHlRUndXbDhqN2hiLVpCR1NpMlJNb0V3LWNURG1KYjIweWUwQXZrWHhqVmxqbTN1aGpWVWRHTEtTQ0dfM1I4V0VuWEI3akRTV3Zpd0NEdDFKLWtPSW5EOEVUcjFvVDJKWWJ5N0FsaS12R25jdjJRdlhSb010RG9MN3F0MmkzSHNNZzhORjFDSHVhRUQ3RXdiTEMwRTRpWnZfcUw2WW45endqMVZ2bUZtbjA3T1ItanVOYkFnUXAtb01XR1lORDFKMnRpSW5QV0RtVUNBd0VBQWFBQU1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQ0FRREdPdjUxc1hlUWNSLVhYMmUtbDZfSEt1WjNfVTdKbTJmNWtMMWJvbkpwOUM0UExacVNZMzNDZE5FbE1BcEVRczFzLTVhWEJCemRYWWE1X05hTFB2cm5fRm5mb2d1cnJHOXV6cU1vT0QtMjMtUnd5QkNLZFpNQ3gyVmd0YWNFU3RiZ2RLamNMRnRNRVE4YnR1NHIxMXVKQWlrblRIQnk4V3ZmaHREVS1Da0FkT2FYZV8zMktKSVV4Z05LSzhiYnRVUGlFc21jd3VqUGVzUkprWUh1QWVKc2JFQkY5ekVZNjlCazZiZVZKUUpxRjR4VjhYYmJheGZSX1N6TG5NWnJZNFhoNDNXbGRPN1UzZm9BZHYtLWk3eTlDbDUxaTJRV1RZMHFGcGVmd19nUU93SFFWMW9BRWJ0OWwyYkgyNGEtZ2NKUE9RNEhTdTBEV0ZHaFdSVkVuMUJsQ01XMkxGQnp2elpzMGdIaFhnQ1psVnNGcE1nYndJMThBLTA4UjZvS2FRWC1fM2tDb0FIaXcxQ1pdanaVQ1ZVOVRZNXNUMVlnZXBJVzBkT0VHYXY3YUJMXzNCbk9HVzVlMlZ2LXN5aGVSZS1ORzhXTEZiOHRyc2hMYTRPOVVjS3h3Nzl0MjFGaEhUYXhIblJLcDhFR3p3M2ZoZElMUW42YVlkb0k4Wm9faGJJaUE0cEhoMXlCbGpLU2Q3Zk1xTzkzX3JxV2Y4NzRfd2Q4N3RhcDFmb1pyZ1dYMVU5Wm9ZUnhFZ0FQOVN1cUdrcTJVUl9ucU9CQl9XaVBPM2ZGcFc3cTB6UEp1QUtBNWZIdDdFRG1HUldkTWNGXzM0SDdNenZPQk4tckI2S3VZTUtzWXpkS1ZEMDhwUnhUVVhKc3Nrb2t2MVF3aGNmNklzdEFtMDJ6bjhfWHBRIn0",
"header": {
"alg": "RS256",
"jwk": {
"e": "AQAB",
"kty": "RSA",
"n": "xVZG_h6B314tV_UNG-KUA_wldRuRjXvdcLwwtzOSBBjA1aGa-wabVUjazf2DrPWHlhiFlfom0sV0JgR2Ak5Ydlr4OOTqWCQ6m-ABnl71DvUs-u8eQwcLPsp-ccmRW3vYGuXoSP7-TEM9MSfAI-jeJ9vXeyDUGQDTD1FcBcZh886tR6LwyHBUbE0aD7I5I6pKr5kn24utnXcQ0LNoTOwjycexwzb-kGYHKfHdK5Chx1XLUkZIw7SYqePTchcBRsn6WOYLZ-orT4G58CRNbqpWa6qeRDijCOguUZfaJPuZLJl8ULIhtim0k1Y2e-X8tCNn-qacraicW6mPdlRcBUXAzQ"
}
},
"protected": "eyJhbGciOiJSUzI1NiIsImp3ayI6eyJlIjoiQVFBQiIsImt0eTI6IlJTQSIsIm4iOiJ4VlpHX2g2QjMxNHRWX1VORy1LVUFfd2xkUnVSalh2ZGNMd3d0ek9TQkJqQTFhR2Etd2FiVlVqYXpmMkRyUFdIbGhpRmxmb20wc1YwSmdSMkFrNVlkbHI0T09UcVdDUTZtLTRMbmw3MUR2VXMtdThlUXdjTFBzcC1jY21SVzN2WUd1WG9TUDctVEVNOU1TZkFJLWplSjl2WGV5RFVHUURURDFGY0JjWmg4ODZ0UjZMd3lIQlViRTBhRDdJNUk2cEtyNWtuMjR1dG5YY1EwTE5vVE93anljZAv3emIta0dZSEtmSGRLNUNoeDFYTFVrWkl3N1NZcWVQVGNoY0JSc242V09ZTFotb3JUNEc1OENTTmJxcFdhNnFlUkRpakPNZ3VVWmZhSlB1WkxKbDhVTElodGltMGsxWTJlLVg4dENObi1xYWNyYWljVzZtUGRsUmNCVVhBelEifSwibm9uY2UiOiJidGY3SFpROHlvVERGNVphWjdaSnVGR05tOWR2cWhyNmdWVHR0NHZYbmFvIn0",
"signature": "Mo1ZVEkT_QjsH4Yy98tTm3JEpsccnriVn5L18yjN2O1ea57V3apkDkkMb_3wleJ0YJskSuNrvtftJOC_-OqeT1_qbq4AjugEqMPle5I7VUAzshnh1DL7YiAgds5Fm06VtCuWUns5owF2MtVmjKMJHdHc9a_9-jilQsFWrTHEZgTt_ebBHazFpiEVcqoNCxhho-XxWZaHlvDOncJXUnqG0SWIa0OeM5Gm80jlPRlQoE5Wp6RqQvn1Fsb3NpzMUEQwD-s9JCvB4U2tQdpGLM5ynfbFwlgyS1AgKiQ4FLEftc55Yo9yOo0bXEugM7aDZS7-_TjqFD_N7r0IJHPp8fXrCQ"
}
This is what's called a JWS (JSON Web Signature), specifically a "JWS Using Flattened JWS JSON Serialization" from RFC 7515. Scary stuff eh π»? Don't worry, we'll break down the anatomy of this strange looking request in the sections below.
One problem we'll run into is that when we sign our payload with our key, we might not get ASCII out, even if we're only putting ASCII in. We can see this for ourselves:
puts client_key.sign OpenSSL::Digest::SHA256.new, 'Hello world'
οΏ½οΏ½ΫοΏ½οΏ½7οΏ½xMοΏ½οΏ½\οΏ½AU=οΏ½KGQοΏ½οΏ½aoοΏ½:Q-HοΏ½WWοΏ½a_ΤοΏ½οΏ½οΏ½οΏ½+a
οΏ½οΏ½οΏ½|X]οΏ½s}VοΏ½oyaοΏ½οΏ½οΏ½'68L6οΏ½οΏ½οΏ½οΏ½PοΏ½οΏ½οΏ½οΏ½οΏ½fοΏ½οΏ½yKVοΏ½οΏ½οΏ½
οΏ½I@οΏ½οΏ½οΏ½aοΏ½οΏ½[οΏ½οΏ½οΏ½οΏ½οΏ½CοΏ½οΏ½οΏ½VXM+οΏ½
οΏ½οΏ½oQοΏ½@οΏ½BοΏ½"]UzrοΏ½NοΏ½R]]{9;οΏ½N:οΏ½οΏ½GοΏ½ΣaMοΏ½SοΏ½οΏ½HοΏ½Ε΅qοΏ½οΏ½οΏ½BqοΏ½9οΏ½οΏ½ οΏ½οΏ½SoοΏ½QοΏ½οΏ½οΏ½tkοΏ½;οΏ½οΏ½οΏ½οΏ½zοΏ½οΏ½dοΏ½<=οΏ½οΏ½ +B
_tοΏ½
οΏ½οΏ½οΏ½οΏ½οΏ½~οΏ½οΏ½οΏ½οΏ½<Λ―ή€
οΏ½%ΓοΏ½kοΏ½οΏ½
To avoid dealing with non-ASCII characters we'll need to Base64 encode most of our data. The good news is Ruby comes with Base64 handling as part of the standard library:
Base64.urlsafe_encode64('test')
#=> "dGVzdA=="
There is a small tweak we'll need to make to keep Let's Encrypt happy - removing the padding characters (=
) from our encoded data:
Base64.urlsafe_encode64('test').delete('=')
#=> "dGVzdA"
(Or in Ruby 2.3)
Base64.urlsafe_encode64('test', padding: false)
Let's wrap that in a helper method - we'll be using it a lot as we build our request:
def base64_le(data)
Base64.urlsafe_encode64(data).delete('=')
end
The payload is the simplest part of our request. It's just JSON that we'll Base64 encode using our method above:
base64_le '{"resource":"new-reg"}'
#=> "eyJyZXNvdXJjZSI6ICJuZXctcmVnIn0"
This a totally valid payload that we can send to Let's Encrypt. Obviously it'll be more convienient not to have to construct JSON strings by hand - so let's load in the JSON library (again part of the Ruby standard lib):
require 'json'
base64_le JSON.dump(resource: 'new-reg')
#=> "eyJyZXNvdXJjZSI6ICJuZXctcmVnIn0"
For further convenience, we can make our Base64 helper method smarter. If the data we pass in is an array or hash, it should JSONify the data before encoding it:
def base64_le(data)
txt_data = data.respond_to?(:entries) ? JSON.dump(data) : data
Base64.urlsafe_encode64(txt_data).delete('=')
end
That's all we need for our payload π! As well as providing information about the request we want to make, it will form one half of the data we'll be signing.
We'll need to give Let's Encrypt two things for it to validate the authenticity of the request: our public key, and the cryptographic hashing algorithm we're using to generate the signature. This info will go in our header.
The static parts of our header are as follows:
header = {
alg: 'RS256',
jwk: {
kty: 'RSA',
}
}
alg
corresponds with the hashing algorithm we want to use - in this case SHA-256 (or more technically RSA PKCS#1 v1.5 signature with SHA-256, but we don't really have to worry about that here). kty
means key type - our keys are RSA keys. jwk
stands for JSON web key - a standard for sharing keys via JSON.
The parts of the key we're interested in are the public key exponent (e) and the modulus (n). Helpfully our client_key
has corresponding methods (client_key.e
and client_key.n
) - the only additionally steps we need to take are converting them to binary strings with to_s(2)
(documented here), then (you guessed it), Base64 encoding them. Let's also create a header
convenience method:
def header
{
alg: 'RS256',
jwk: {
e: base64_le(client_key.e.to_s(2)),
kty: 'RSA',
n: base64_le(client_key.n.to_s(2)),
}
}
end
We have our plaintext header - which contains the required components of our public key. We'll also need a protected header - basically a Base64 encoded version of our header which will form the other half of the data we'll be signing (alongside our payload).
The protected header contains one additional element which our unprotected header doesn't - a cryptographic nonce. The linked article goes into lots of details, but a nonce is basically a one-time use code which we must attach to our request. It means if an attacker somehow sniffs out a request we made, and makes a carbon-copy duplicate request, the attackers attempt will fail (because the nonce has already been used).
Let's Encrypt provides us a nonce in the headers of every response it gives us - so getting a nonce is just a case of requesting any Let's Encrypt API endpoint, and grabbing it from the Replay-Nonce
header.
Ruby comes with the Net::HTTP
library built in for making HTTP requests, but it's a bit cumbersome. To make our life easier, we'll use HTTParty - although this is by no means a necessity.
gem install httparty
require 'httparty'
(Note: you can also grab the Gemfile
provided in this repository, and bundle install
to save yourself some typing.)
We'll send HTTParty's debug output to $stdout
so we can see easily see the requests/responses happening:
HTTParty::Basement.default_options.update(debug_output: $stdout)
As mentioned above, any Let's Encrypt API endpoint will do - let's use the /directory
endpoint (effectively the root of the API). Because we only need the headers, we can just make a HEAD
request:
nonce = HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']
We can now create our protected header:
protected = base64_le(header.merge(nonce: nonce))
The last step to construct our request is proving its authenticity with a signature, generated using our client private key. First, let's consolidate everything we have so far:
require 'openssl'
require 'base64'
require 'json'
require 'httparty'
def base64_le(data)
txt_data = data.respond_to?(:entries) ? JSON.dump(data) : data
Base64.urlsafe_encode64(txt_data).delete('=')
end
client_key_path = File.expand_path('~/.ssh/id_rsa')
client_key = OpenSSL::PKey::RSA.new IO.read(client_key_path)
payload = { some: 'data'}
header = {
alg: 'RS256',
jwk: {
e: base64_le(client_key.e.to_s(2)),
kty: 'RSA',
n: base64_le(client_key.n.to_s(2)),
}
}
nonce = HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']
request = {
payload: base64_le(payload),
header: header,
protected: base64_le(header.merge(nonce: nonce))
}
As mentioned above, we'll be using the SHA-256 hash function for our signing:
hash_algo = OpenSSL::Digest::SHA256.new
The specific data we'll need to sign is our protected header and our payload, joined with a period:
request[:signature] = client_key.sign(hash_algo, [ request[:protected], request[:payload] ].join('.'))
Now we've built the request data just as Let's Encrypt wants, we have everything we need to start making requests:
HTTParty.post(some_api_endpoint, body: JSON.dump(request))
Let's put everything into a reusable method that can take an arbitrary URL and payload. (We'll move client_key
, hash_algo
, header
and nonce
into methods at the same time):
HTTParty::Basement.default_options.update(debug_output: $stdout)
def client_key
@client_key ||= begin
client_key_path = File.expand_path('~/.ssh/id_rsa')
OpenSSL::PKey::RSA.new IO.read(client_key_path)
end
end
def header
@header ||= {
alg: 'RS256',
jwk: {
e: base64_le(client_key.e.to_s(2)),
kty: 'RSA',
n: base64_le(client_key.n.to_s(2))
}
}
end
def hash_algo
OpenSSL::Digest::SHA256.new
end
def nonce
HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']
end
def signed_request(url, payload)
request = {
payload: base64_le(payload),
header: header,
protected: base64_le(header.merge(nonce: nonce))
}
request[:signature] = client_key.sign(hash_algo, [ request[:protected], request[:payload] ].join('.'))
HTTParty.post(url, body: JSON.dump(request))
end
The /directory
endpoint that we use to fetch our nonce serves another purpose: it lists all the other endpoints which act as the starting points for all the core actions (registering a user, authorizing a domain, issuing a certificate etc.):
{
"key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
"meta": {
"terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf"
},
"new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
"new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
"new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
"revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}
(Note: unlike the API's endpoints, the directory is viewable without any kind of signing, you can just visit it in your browser).
Here the keys in the JSON object indicate the resource type, and the values are the URI we'll need to make a signed request to. Even though Cool URIs don't change, using the directory means we don't have to hard-code the endpoints - and so our client is more resilient to any changes Let's Encrypt might make (credit to @kelunik for suggesting this).
To avoid making repeated requests to the directory, let's make an endpoints
method:
def endpoints
@endpoints ||= HTTParty.get('https://acme-v01.api.letsencrypt.org/directory').to_h
end
Since we're referencing the directory endpoint in both our endpoints
and nonce
methods, we can dry up our code by moving it into a constant. This will also make it easier to, for example, switch to LE's staging server.
DIRECTORY_URI = 'https://acme-v01.api.letsencrypt.org/directory'.freeze
def nonce
HTTParty.head(DIRECTORY_URI)['Replay-Nonce']
end
def endpoints
@endpoints ||= HTTParty.get(DIRECTORY_URI).to_h
end
The neat thing is that this DIRECTORY_URI
is the only URI we need to hardcode, every other endpoint we can either pull from the directory, or from the Location
or Link
headers of the API's responses. Location
is easy to work with (it's just a single URI) - but Link
headers need to be parsed. I've written a gem (Nitlink) for just that - so let's install and load it:
gem install nitlink
require 'nitlink/response'
OK, we've laid the foundations - let's make our first actual request to the Let's Encrypt API! The first step is to register our client public key with Let's Encrypt.
Since we're sending the public key with every request (in the header
property of our JSON), we don't need to include much in the actual registration payload. At a minimum, we'll just need to specify the resource type: new-reg
in this case.
new_registration = signed_request(endpoints['new-reg'], {
resource: 'new-reg',
})
We can optionally provide contact details (highly recommended), this will allow us to recover our key in case we lose it. We'll need to include the protocols for the contact details we provide - e.g. mailto:
for email addresses, (it turns out Let's Encrypt doesn't currently support this, see here).tel:
for phone numbers
new_registration = signed_request(endpoints['new-reg'], {
resource: 'new-reg',
contact: ['mailto:[email protected]']
})
Sending the request should give us back a successful response:
-> "HTTP/1.1 201 Created"
-> "Content-Type: application/json"
-> "Location: https://acme-v01.api.letsencrypt.org/acme/reg/12345"
-> "Link: <https://acme-v01.api.letsencrypt.org/acme/new-authz>;rel=\"next\", <https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf>;rel=\"terms-of-service\""
...
{
"id": 12345,
"key": {
"kty": "RSA",
"n": "wlpAF2eAhpzJDGCco-c9hhd31NGAyhkFeivqfmt7ZQiphRiuSwF_0_3lOnCRpdpRIeVheIPVK6FofcFVmRjzdyDeZmN5ssk5oi2v1y8hSB7SM2QCoqlZ3L8uEGKzzwQfzSIQGIR56X5GrTKaCjBrzqrSM0VzRg5-gp8ZDqsyceSUaf7SgScxexfgbcaRXtJ1aVLYT5FfsDgV768gRcBxaKQapFQ47M7JN8OTOq6QIla6acp24eNo6PMtH8Mf0hJwpcWOs2A_0VcNzV7XBl8shYEeERyqbNXIZsF7njF8WInk7-v0EiYPV2w0xjBuFnbX7cw8YqveG81yirYGScR5ASeER5dxtWNyXFXkK9KpI13Vvf-0ivzrgeJTUsKz7EAjL2vof2QleKZHjP6f63rvaIMK5FaGojhHSzzMdeP3FaG1mP7N5vY3J0oZzhny_Jd9vNysCiklsUNUr8ZT-ocTKHbiO6ZEZdj8Wtjmpr5kvfPUtosNodaMUNFv-7UFRWNf49qJKo21UzpeeM7Us0hKPNVd9VU0qD0jsya7w1EjimiBqwo6vD_KoH-R2bwWlaQ9Ucy6ahfNPogI3zqMTpUfMXGA0uMj6anp-daOSwuEus2ogY0x12OUn3XivB9VzbCNadAT9JqKRrhRHE-7tfN6TFt7CtLjGCe1ShMn3wsMFBU",
"e": "AQAB"
},
"contact": ["mailto:[email protected]"],
"initialIp": "101.222.66.199",
"createdAt": "2015-12-12T12:07:23.755314388Z"
}
The successful response basically just echoes back to us our registration details. We can see the exponent + modulus (e
and n
) values of our public key included at the top, as well as the unique id
of our new account.
Note that LE verifies the domains of emails we provide (by checking their DNS A
record), so make sure it's a real domain, otherwise you'll get an 400 (Bad Request) response:
{
"type": "urn:acme:error:malformed",
"detail": "Error creating new registration :: Validation of contact mailto:[email protected] failed: Server failure at resolver",
"status": 400
}
If we try and register the same key again we'll get a 409 (Conflict) response:
-> "HTTP/1.1 409 Conflict"
-> "Content-Type: application/problem+json"
-> "Location: https://acme-v01.api.letsencrypt.org/acme/reg/12345"
...
-> "Connection: close"
{
"type": "urn:acme:error:malformed",
"detail": "Registration key is already in use",
"status": 409
}
Don't worry, there are no side effects to attempting to re-register the same client key multiple times
Although we've successfully registered, Let's Encrypt won't let us do anything useful (like issue a certificate), until we accept their Subscriber Agreement.
To indicate our acceptance, we just need to make a request to the URI of our newly created user (returned in the response's Location
header, in this case https://acme-v01.api.letsencrypt.org/acme/reg/12345
) with the payload's agreement
key set to the URI of the terms we're accepting. How do we know the URI of the terms? Eagle-eyed readers might have spotted above that it's returned as one of the response's Link
headers:
Link: ... <https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf>;rel="terms-of-service"
Update August 2017: The URL of the current terms are now also available through the directory, and thus via our endpoints
method, specifically endpoints['meta']['terms-of-service']
.
We should also check that we got a 201 status (not a Conflict or malformed registration). Our final code for accepting the terms programatically looks like this:
if new_registration.status == 201
signed_request(new_registration.headers['Location'], {
resource: 'reg',
agreement: new_registration.links.by_rel('terms-of-service').target
})
end
(The .links
method depends on the Nitlink gem, we'll get a NoMethodError
if it's not installed). Notice that the resource type has changed, since we're not creating a new user, but modifying an existing one. Also, a real client should probably prompt the user to actually read the agreement - rather than just auto-accepting it π!
The next step is to inform Let's Encrypt which domain or subdomain we to provision a certificate for. In this guide I'm using the example le.alexpeattie.com. This is the first part of a multistep verification process to prove we're the legitimate owner of the domain:
- a). We ask LE for the challenge
- b). LE gives us a challenge to prove we control the domain
- c) or d). We complete the HTTP- or DNS-based challenge, and notify LE that we're ready
- e). LE checks the challenge has been completed to it's satisfaction
- f). We verify that LE is happy the challenge has been passed π
Challenges are how we prove a sufficient level of control over the identifier (domain name) in question. We can do this either by serving a specific response when LE hits a specific URL (which generally means uploading a file to our web-server), provisioning a DNS record, or by leveraging the Server Name Indication extension of TLS to serve a special self-signed certificate.
We'll cover the first two kinds of challenge: http-01
and dns-01
but not the third (tls-sni-01
).
Asking LE for our new challenge is just a case of making another request to the LE API - this time to create a new-authz
(authz
is short for authorization).
As you can probably guess, this means making a request to the new-authz
endpoint with our resource
option set to (you guessed it) new-authz
. Beyond that just need tell LE what identifier (domain name) we want to authorize:
auth = signed_request(endpoints['new-authz'], {
resource: 'new-authz',
identifier: {
type: 'dns',
value: 'le.alexpeattie.com'
}
})
The ACME spec is designed to be flexible enough to authorize more than just domain names in the future - which is why we have to explicitly state we're authorizing a domain name with type: 'dns'
. We could authorize the root domain with value: 'alexpeattie.com'
. We can also provide a Punycode encoded IDN, see Appendix 6.
Let's Encrypt should send up back a nice meaty response like the below π -
{
"identifier": {
"type": "dns",
"value": "le.alexpeattie.com"
},
"status": "pending",
"expires": "2016-01-15T19:28:33.644298086Z",
"challenges": [{
"type": "tls-sni-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157173",
"token": "rsFpjtnLgfXS8hMrAAcSsXJ98q7YNlA2Iyky-EWmoDY"
}, {
"type": "http-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157174",
"token": "w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10"
}, {
"type": "dns-01",
"status": "pending",
"uri": "https: //acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157175",
"token": "U-85Krl7E2bPhqhdrjTuBoeIc7IVJ7Z4wyUhhn0uij0"
}],
"combinations": [
[0],
[2],
[1]
]
}
Let's break this down. First our "identifier"
is echoed back to us, along with its "status"
- right now it's "pending"
which means we haven't proven to LE that we control the domain; we're aiming to change it to "valid"
. Our challenge also has an expiry date - 1 week from now at the time of writing.
http_challenge, dns_challenge = ['http-01', 'dns-01'].map do |challenge_type|
auth['challenges'].find { |challenge| challenge['type'] == challenge_type }
end
The "uri"
of the challenge will allow us to notify LE that we're ready to take the challenge, on to check if we've passed. The "token"
is a unique, unguessable, random value sent to us by LE that we'll need to incorporate into our challenge response to prove we control the domain.
"combinations"
is a another feature that's designed for the future. Right now we only have to pass 1 challenge to convince LE we control the domain. In the future we might see something like this:
"challenges": [{
"type": "email-01",
"..."
}, {
"type": "http-01",
}, {
"type": "tls-sni-01",
}, {
"type": "dns-01",
}],
"combinations": [
[0, 1],
[2],
[3]
]
Which would mean we'd have to either pass both challenges 0 & 1 (the "email-01"
and "http-01"
challenges), or challenge 2 or challenge 3 ("tls-sni-01"
or "dns-01"
).
Our first option is the http-01
challenge. To pass this we need to ensure that when LE makes a request to
https://<< Domain >>/.well-known/acme-challenge/<< Challenge token >>
They receive a specific response (more on that below). Our domain is le.alexpeattie.com
, .well-known/acme-challenge/
is a fixed path defined by ACME, and our challenge token is w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10
, so the endpoint we'll need to serve the response from is:
https://le.alexpeattie.com/.well-known/acme-challenge/w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10
First we'll create our key authorization: the special response LE expects to be served. It's quite simple - it's the challenge token and a 'thumbprint' of our public key joined with a period.
We're using the JSON Web Key standard to share details of our public key already (in the jwk
field of our header). To generate the thumbprint we need to generate a digest of that JSON using SHA256
, and Base64 encode it (see RFC 7638 for more).
Our final code for the thumbprint
method looks like this:
def thumbprint
jwk = JSON.dump(header[:jwk])
thumbprint = base64_le(hash_algo.digest jwk)
end
And for our final challenge response:
http_challenge_response = [http_challenge['token'], thumbprint].join('.')
To prove to LE that we control a domain, https://example.com/.well-known/acme-challenge/<< Challenge token >>
needs respond with << Challenge token >>.<< JWK thumbprint >>
. Because this is just a toy client, we'll create the file locally, then upload it (using SCP) to our remote nginx server - a more usual approach would be to run the LE client on the server (so we can just write the necessary files directly to disk).
We'll use the net-scp
gem for easier SCP uploads:
gem install net-scp
Since we're serving static files with nginx from /usr/share/nginx/html
, so we'll first want to create the .well-known/acme-challenge
directory:
ssh [email protected] 'mkdir -p /usr/share/nginx/html/.well-known/acme-challenge'
The code for uploading the challenge is quite straightforward:
require 'net/scp'
def upload(local_path, remote_path)
server_ip = '162.243.201.152' # see Appendix 3
Net::SCP.upload!(server_ip, 'root', local_path, remote_path)
end
# ..
destination_dir = '/usr/share/nginx/html/.well-known/acme-challenge/'
IO.write('challenge.tmp', http_challenge_response)
upload('challenge.tmp', destination_dir + http_challenge['token']) and File.delete('challenge.tmp')
Our simple nginx setup (see Appendix 3) serves static files (if they exist) for any endpoint, so this should be all we need to ensure that a request to https://le.alexpeattie.com/.well-known/acme-challenge/w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10
returns our key authorization as its response (we can verify this in a browser).
The dns-01
challenge was introduced at the beginning of 2016, allowing us to authorize our domain(s) by provisioning DNS records. The key differences between the http-01
challenge and the dns-01
challenge are:
- We'll add a DNS TXT record rather than uploading a file
- Rather than using "raw" key authorization as the record's contents, we'll use its (Base64 encoded) SHA-256 digest (see below)
There are lots of ways to add the required DNS record - most DNS services provide a web interface (instructions for common providers here) - we'll be programatically adding a record using the DNSimple API & associated gem.
The key ingredients of a DNS record are its type, name and value/contents. The type of the record is TXT
, which is designed for adding arbitrary text data to a DNS zone. The name of the record takes the format _acme-challenge.subdomain.example.com
. The root domain name is appended to a record's name automatically, so we just need to provide the name as _acme-challenge.subdomain
or just _acme-challenge
if we're authorization the root domain.
record_name = '_acme-challenge.le'
To construct the contents of our record, we'll start by creating our "raw" challenge response in the same manner as in the http-01
challenge:
raw_challenge_response = [dns_challenge['token'], thumbprint].join('.')
Additionally, for the dns-01
we'll need to digest the challenge response, and run it through our base64_le
method:
dns_challenge_response = base64_le(hash_algo.digest raw_challenge_response)
We'll use the dnsimple-ruby gem to add our TXT
record:
gem install dnsimple -v 2.2
We'll also need to get our API token from the DNSimple web interface. Then using the gem to add the TXT record, with the correct record name & content. We'll set a relatively low TTL (time to live) of 60 seconds, because we don't want our resolvers to cache the record for long - in case we need to redo the challenge, for example.
require 'dnsimple'
dnsimple = Dnsimple::Client.new(username: ENV['DNSIMPLE_USERNAME'], api_token: ENV['DNSIMPLE_TOKEN'])
challenge_record = dnsimple.domains.create_record('alexpeattie.com', {
record_type: 'TXT',
name: record_name,
content: dns_challenge_response,
ttl: 60
})
(Note: DNSimple is growing and at the time this guide created we're using DNSimple API v1. The DNSIMPLE_USERNAME
is the email address linked to your DNSimple account and DNSIMPLE_TOKEN
is API v1 token.)
Lastly, we'll use Ruby's Resolv library (part of the std lib) to wait until the challenge record's been added:
loop do
resolved_record = Resolv::DNS.open { |r| r.getresources(record_name + '.alexpeattie.com', Resolv::DNS::Resource::IN::TXT) }[0]
break if resolved_record && resolved_record.data == challenge_response
sleep 5
end
To tell LE we've completed the challenge, we need to make a request to the challenge URI we got earlier (https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157174
or /5157175
).
Our request needs to include the field keyAuthorization
with the key authorization we've just generated:
signed_request(http_challenge['uri'], { # or dns_challenge['uri']
resource: 'challenge',
keyAuthorization: http_challenge_response # or dns_challenge_response,
})
Finally it's just a case of polling the challenge URI we've been given and wait for its status to become "valid"
. If it's still "pending"
we'll sleep
for 2 seconds then try again. Any other status means something's gone wrong π.
loop do
challenge_result = HTTParty.get(challenge['uri']) # or dns_challenge['uri']
case challenge_result['status']
when 'valid' then break
when 'pending' then sleep 2
else raise "Challenge attempt #{ challenge_result['status'] }: #{ challenge_result['error']['details'] }"
end
end
If we chose the DNS challenge, we should also clean up after ourselves by deleting the record (so our challenge attempt doesn't interfere with future challenge attempts, which will also require TXT
records using the _acme-challenge.le
name):
dnsimple.domains.delete_record('alexpeattie.com', challenge_record.id)
We've proven to Let's Encrypt we control the domain, which means we can now get our certificate. We'll need to generate a Certificate signing request (CSR). The CSR includes the public part of the key-pair tied to the certificate - secure traffic will be encrypted with the corresponding private part of the key-pair.
It's best to create a new key-pair for our CSR. We can generate it on the command line (as for the client key-pair), or with Ruby:
domain_key = OpenSSL::PKey::RSA.new(4096)
IO.write('domain.key', domain_key.to_pem)
*You might alternatively want to use a 2048 bit key (see Appendix 5 for more).
Ruby's OpenSSL
module makes the generation of the CSR very straightforward:
csr = OpenSSL::X509::Request.new
csr.subject = OpenSSL::X509::Name.new([['CN', 'le.alexpeattie.com']])
csr.public_key = domain_key.public_key
csr.sign domain_key, hash_algo
We need to set the subject of the CSR - in this case the common name (domain name) we want to secure. Then we sign our certificate with our (private) domain_key
.
LE needs us to send CSR in binary (.der) format - Base64 encoded of course. We'll be making a request for a new-cert
:
certificate_response = signed_request(endpoints['new-cert'], {
resource: 'new-cert',
csr: base64_le(csr.to_der),
})
Let's Encrypt should respond with our brand new, DV certificate π π. It's not quite ready to use though.
Certificates should be typically enclosed by a -----BEGIN CERTIFICATE-----
header and -----END CERTIFICATE-----
(RFC here) with each line wrapped at 64 characters. We could either do that manually (e.g. see tiny-acme's implementation) or let OpenSSL::X509::Certificate
take care of it:
certificate = OpenSSL::X509::Certificate.new(certificate_response.body)
We also need to complete our trust chain, which means grabbing the LetsEncrypt cross-signed intermediate certificate (see https://letsencrypt.org/certificates/). Some browsers will resolve an incomplete trust chain, but it's something we want to avoid. There's much more info on why we need to complete this step and the difference between the different intermediates LE offers in Appendix 2: The trust chain & intermediate certificates.
Occasionally server software might want us to provide our intermediate certificates separately, but generally we'll bundle them together in a single file. Helpfully, LE provides a link to the latest intermediate certificate in the certificate response's Link
header (it has the relation type "up"
):
Link: </acme/issuer-cert>;rel="up"
intermediate = OpenSSL::X509::Certificate.new HTTParty.get(certificate_response.links.by_rel('up').target).body
IO.write('chained.pem', [certificate.to_pem, intermediate.to_pem].join("\n"))
That's it - we're done with our client and have our certificate (valid for the next 90 days) that will be accepted by all major browsers β ! Completed authorizations are valid for 300 days, so we can renew our certificate without needing to take a challenge during that period.
This is the end of the main part of the guide, if you're interesting in the logistics of installing the certificate, keep reading...
Now we have our certificate, it's just a case of uploading it along with our private key and tweaking our nginx configuration to enable TLS. As with our HTTP challenge response, we can upload the necessary files with SCP using our upload
helper method:
upload('chained.pem', '/etc/nginx/le-alexpeattie.pem')
upload('domain.key', '/etc/nginx/le-alexpeattie.key')
Then we'll need to point our nginx.conf
to our certificate and key:
server {
listen 443 ssl deferred;
server_name le.alexpeattie.com;
ssl_certificate /etc/nginx/le-alexpeattie.pem;
ssl_certificate_key /etc/nginx/le-alexpeattie.key;
}
That's theoretically all we need, but we can improve on nginx's defaults for better security and performance. We'll use the settings recommended by https://cipherli.st/ (click "Yes, give me a ciphersuite that works with legacy / old software." if you need to support older browsers) and a couple of extra headers recommended by securityheaders.io. We'll use Google's DNS server (8.8.8.8) as our resolver
(recommended for OSCP stapling on nginx):
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8;
resolver_timeout 5s;
# add_header Strict-Transport-Security "max-age=63072000; preload" always;
add_header X-Frame-Options DENY always;
add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Content-Security-Policy "default-src https: data: 'unsafe-inline' 'unsafe-eval'" always;
(Note: that the Content-Security-Policy
header will prevent assets being loaded over HTTP - this is recommended, but could break some sites. Read more about CSPs here)
We should keep the line enabling the Strict-Transport-Security
header commented out until we're happy our HTTPS setup is working (as visitor's won't be able to access our non-HTTPS site once it's activated).
We can harden our configuration by dropping support for TLS < v1.2 - although that does have implications for supporting older browsers. If we happy to target just older browsers, we should also allow only cipher suites with a minimum 256-bit key length for AES (the symmetric cipher):
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "AES256+EECDH:AES256+EDH";
Unless we're using an ECDSA certificate (see Appendix 5) we should also generate a stronger DH parameter - nginx uses a 1024-bit prime which has been shown to be potentially vulnerable to state level adversaries (https://weakdh.org). Ideally our DH parameter shouldn't be smaller than our key size (i.e. 4096-bit or 2048-bit). We can generate a DH parameter like so:
ssh [email protected]
cd /etc/nginx
openssl dhparam -out dhparam.pem 4096
Bear in mind, the above is slooow (it took about 30 minutes for me) - so an alternative is to take a pre-generated prime from here:
curl -o dhparam.pem https://2ton.com.au/dhparam/4096/`shuf -i 0-127 -n 1`
openssl dhparam -in dhparam.pem -noout -text | head -n 1
#=> PKCS#3 DH Parameters: (4096 bit)
Either way we'll need to tell nginx to use our stronger DH parameter:
ssl_dhparam /etc/nginx/dhparam.pem;
Lastly, we can redirect all HTTP traffic to our HTTPS endpoint:
server {
listen 80;
server_name le.alexpeattie.com;
return 301 https://$host$request_uri;
}
Our final nginx.conf
looks like this:
events {
worker_connections 1024;
}
http {
sendfile on;
server_tokens off;
root /usr/share/nginx/html;
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
server {
listen 443 ssl deferred;
server_name le.alexpeattie.com;
ssl_certificate /etc/nginx/le-alexpeattie.pem;
ssl_certificate_key /etc/nginx/le-alexpeattie.key;
ssl_dhparam /etc/nginx/dhparam.pem;
ssl_trusted_certificate /etc/nginx/le-alexpeattie.pem;
}
server {
listen 80;
server_name le.alexpeattie.com;
return 301 https://$host$request_uri;
}
}
Lastly let's run some tests to ensure our certificates are correctly and securely installed. There are a few tools out there, Qualys SSL Server Test is the most widely used. Using our new certificate with the strict cipher list, with either an ECDSA certificate or a standard certificate with a 4096-bit DH param we'll net top marks with a perfect A+ score:
Using cipherli.st's recommended ciphers, we'll score fractionally lower, with 90 points Cipher Strength:
We also do well on securityheader.io test:
As the report points out, we can harden our set up even further by implementing HTTP Public Key Pinning which could protect us if, for instance, Let's Encrypt itself was successfully attacked. However, this is currently considered quite advanced: as Peter Eckersley warns "HPKP pinning carries an inherent risk of bricking your site". But he does give some detailed best practices for the brave souls who do want to implement it.
Some other useful testing tools:
- revocationcheck.com - Useful for debugging OSCP
- testssl.sh and cipherscan - Command line TLS testing tools
- SSL Decoder - Open-source tool for checking SSL/TLS config - gives lots of info, but no score per se.
- High-Tech Bridge SSL Server Security Test - A decent alternative to SSL Labs's tool. Advocates weaker ciphers because of HIPAA guidance though.
The trusted status of a certificate (what gives us the green padlock) stems from a relatively small set of trusted Certificate Authorities (CAs) with corresponding "Root certificates". These are stored in the "trust stores" of browsers or operating systems. We can see Mac OS's trusted roots by going to Keychain Access -> System Roots for example:
If our certificate has been issued by a trusted CA (in our trust store) that certificate is trusted. If the CA isn't in our trust store, we can check if certificate of that CA was issued by a trusted root CA. A certificate issued by a CA, issued by another CA which was issued by a trusted CA is trusted, and so on. The trust chain can involve as many untrusted or "intermediate" CAs as we want, as long as it ultimately goes back to a trusted (root CA).
If it's still unclear, imagine Alice & Bob are having a birthday party. Guests who are invited by Alice or Bob can in turn invite other guests - those guests can invite other guests and so on. At the party, only guests who can prove their invitation leads back to Alice and Bob are trusted:
- Carol β invited by Bob = trusted β
- Doug β invited by Steve β invited by Alice = trusted β
- Fred β invited by Gerard β invited by Eve = untrusted β
At the party, a guest would have to provide information so we could verify the chain of invites led back to Alice or Bob. In the same way, a certificate should provide information about the chain of certificates (called the trust chain) which lead back to a trusted root CA.
In the future Let's Encrypt hopes to have its own trusted root CA: ISRG Root X1. Right now ISRG Root X1 isn't trusted by any browsers or operating systems - so IdenTrust is acting as their root CA instead. Let's Encrypt's intermediate CA is issued directly by Identrust's root CA (which is trusted by all major browsers/OSes) so our trust chain is only three links long:
- Our certificate β issued by Let's Encrypt CA β 'issued' by Identrust CA
'Issued' is a bit of oversimplification here - in fact, Identrust just cross-signed LE's CA certificate, but it achieves the same end-result: trust in all major browsers/OSes.
So our complete trust chain should include our certificate, the certificate of Let's Encrypt's intermediate CA (Letβs Encrypt Authority X3), and optionally the Identrust CA's trusted root certificate. In reality there's no point making the client download the root certificate - it needs to already be in the trust store anywhere. As RFC 2246 says:
Because certificate validation requires that root keys be distributed independently, the self-signed certificate which specifies the root certificate authority may optionally be omitted from the chain, under the assumption that the remote end must already possess it in order to validate it in any case.
So basically we just need to concatenate our certificate with Let's Encrypt CA's certificate and we have a complete chain of trust* π.
FF 44 | Chrome 48 | IE 11 | Safari 7.1 | iOS 8 (Safari) | Windows Phone 8.1 | Android 6 |
---|---|---|---|---|---|---|
β | β | β | β | β | β | β |
*Some servers (like Apache) might want us to provide the our certificate and the rest of the trust chain separately. In this case the rest of the chain would just be the LE intermediate certificate.
If we only provide our certificate without LE's intermediate certificate, we have a broken chain of trust. Most browsers can actually recover from this. LE certificates leverage Authority Information Access which embeds information about the trust chain even if we (system admins) forget to provide it.
We shouldn't rely on this though, most mobile browsers don't support AIA - nor does Firefox (who have explicitly said they won't be adding it).
Here's the result you'll get without providing the intermediate certificate:
FF 44 | Chrome 48 | IE 11 | Safari 7.1 | iOS 8 (Safari) | Windows Phone 8.1 | Android 6 |
---|---|---|---|---|---|---|
β | β | β | β | β | β | β |
Let's Encrypt has it's own root certificate authority that's separate from Identrust, called ISRG Root X1. When this root CA is widely trusted, expect it to take Identrust's place in the trust chain. We can already use an intermediate certificate issued by ISRG Root X1 - the problem is that, at the time of writing, ISRG Root X1 isn't trusted anywhere (to my knowledge).
If we try using the LE root-signed intermediate now, most browsers that support AIA will fallback to the valid trust chain, except desktop Safari.
FF 44 | Chrome 48 | IE 11 | Safari 7.1 | iOS 8 (Safari) | Windows Phone 8.1 | Android 6 |
---|---|---|---|---|---|---|
β | β | β | β | β | β | β |
Below are the instructions to recreate the site setup used as the exemplar in this guide. You'll need:
- A domain name you control
- A DNSimple account (from $5/month, 30 day trial)
- A DigitalOcean droplet (from $5/month)
Digital Ocean has good instructions that cover common registrars. We'll want to point the nameservers to ns1.dnsimple.com
, ns2.dnsimple.com
, ns3.dnsimple.com
and ns4.dnsimple.com
. You'll need to copy over any existing records from your previous DNS provider.
First we'll need to create our droplet. We'll use a $5/month Ubuntu droplet:
We'll also want to add our local machine's SSH key(s). We want to paste the public part of our key (e.g. cat ~/.ssh/id_rsa.pub
):
Once our machine has been provisioned, take a note of the public IP, in this case 162.243.201.152:
Using the IP, we'll SSH into our new box and install nginx:
ssh [email protected]
add-apt-repository ppa:nginx/stable
apt-get update
apt-get install nginx
A configuration like the below will be sufficient for passing the challenges - we'll update it when we actually install our certificate. This needs to go in /etc/nginx/nginx.conf
:
events {
worker_connections 1024;
}
http {
sendfile on;
server_tokens off;
root /usr/share/nginx/html;
server {
listen 80;
server_name le.alexpeattie.com;
}
}
Lastly we'll restart nginx:
sudo service nginx restart
Log in to DNSimple, go to Domains and hit DNS in the sidebar:
The click + Manage records. We want to add an A record:
We'll need to enter the name (our subdomain le
) and set Address to our droplet's Public IP:
We should be ready to go, and the domain (e.g. <le.alexpeattie.com>) should serve the default nginx welcome page. We might have to wait a while for our DNS changes to propagate.
Once we've been issued our certificate, we can install it following the steps in Appendix 1.
Let's Encrypt can issue a single certificates which cover multiple, using the SubjectAltName extension. At the time of writing, Let's Encrypt supports a maximum of 100 SANs per certificate (full LE rate limits are detailed here).
LE has quite conservative per-domain rate limits right now (20 distinct certificates per domain per week) - so using SANs is crucial if you have lots of subdomains to secure*.
A common use-case is having a single certificate cover the naked domain and www.
prefix. We have to authorize both domains; LE doesn't take it for granted that if we control the root domain we also control the www.
subdomain or vice-versa.
domains = %w(example.com www.example.com)
domains.each do |domain|
auth = signed_request(endpoints['new-authz'], {
resource: 'new-authz',
identifier: {
type: 'dns',
value: domain
}
})
#.. rest of challenge passing code
end
Once we've authorized all the subdomains we want to include in the certificate, we can modify our CSR to use the SAN extension (warning: not the prettiest or most readable code you'll ever see):
alt_names = domains.map { |domain| "DNS:#{domain}" }.join(', ')
extension = OpenSSL::X509::ExtensionFactory.new.create_extension('subjectAltName', alt_names, false)
csr.add_attribute OpenSSL::X509::Attribute.new(
'extReq',
OpenSSL::ASN1::Set.new(
[OpenSSL::ASN1::Sequence.new([extension])]
)
)
That's all you need to get certificates to cover multiple host names, you can find the full code of the example in multiple_subdomains.rb
.
*If you're running a site that, say, assigns thousands of subdomains to end users, you may be out of luck since "you can [only] issue certificates containing up to 2,000 unique subdomains per week" (source). The only current work-around is to get your domain added to Public Suffix list - which LE treats as a special case. Additionally, wildcard certificates will launch in January 2018.
Broadly-speaking key size means how hard a key is to crack. Longer keys offer more security, but their bigger size leads to a somewhat slower TLS handshake.
We don't have a very broad choice when it comes to choosing key size. 2048 bits has effectively been an enforced minimum since the beginning of 2014; 4096 bits is the upper bound. 4096 bits is favored by some, but is far from the standard right now. It's anticipated that 2048-bit keys will be considered secure until about 2030.
2048 is the default key size for cerbot. But you will need a 4096 bit key to score perfectly on the Key SSL Labs' test, and there are lively discussions advocating the LE default be raised to 4096 or 3072. CertSimple did an awesome, detailed rundown of the benefits of different key sizes, and basically concluded "it depends".
We will need a key size of 4096 bits to get a perfect SSL Labs score. Not all cloud providers support key sizes above 2048 bits though, AWS CloudFront being a notable example. If you want or need to use a 2048-bit key, you can specify the key length like so:
domain_key = OpenSSL::PKey::RSA.new(2048)
If you really care about picking a good key, you might not want to use RSA at all. ECDSA (Elliptic Curve Digital Signature Algorithm) which gives a much better size vs. security trade-off. A 384 bit ECDSA is considered equivalent to a 7680 bit RSA key, and will also give a perfect SSL Labs score. More importantly, a number recently discovered SSL vulnerabilities (DROWN, Logjam, FREAK) target RSA-specific vulnerabilities which are not present in ECDSA certificates.
We'll have to a bit more work to create an ECDSA CSR (see this blog post I wrote for a more detailed explanation):
# monkey patch to fix https://redmine.ruby-lang.org/issues/5600
OpenSSL::PKey::EC.send(:alias_method, :private?, :private_key?)
domain_key = OpenSSL::PKey::EC.new('secp384r1').generate_key
IO.write('domain.key', domain_key.to_pem)
csr = OpenSSL::X509::Request.new
csr.subject = OpenSSL::X509::Name.new(['CN', 'le.alexpeattie.com'])
csr.public_key = OpenSSL::PKey::EC.new(domain_key)
csr.sign domain_key, OpenSSL::Digest::SHA256.new
ECDSA is pretty well supported: Windows Vista and up, OS X 10.9, Android 3 and iOS 7*
*Source: CertSimple: What web developers should know about SSL but probably don't
Since October 2016 Let's Encrypt has supported Internationalized Domain Names (IDNs). When providing an IDN as the identifier
's value
in our new-authz
request, and when setting the subject of the CSR, we need to use the Punycode representation of the IDN. For example, mΓΌller.de
would become xn--mller-kva.de
.
You can do the conversion with an online service like Punycoder or with a gem like SimpleIDN:
require 'simpleidn'
domain = SimpleIDN.to_unicode('mΓΌller.de')
=> 'xn--mller-kva.de'
As well as support ECDSA-based certificates (see above), since 2016 LetsEncrypt has supported ECDSA for client (A.K.A account) keys. We'll have to make a few non-trivial modifications to our client to get EC client keys working though.
First, we'll need an EC keypair:
openssl ecparam -genkey -name secp256k1 -noout -out ec-private.pem
openssl ec -in ec-private.pem -pubout -out ec-public.pem
(Alternatively you can grab an Boulder's test EC key here
Then, we'll need to change our client_key
method to load our EC private key.
OpenSSL::PKey::EC.new IO.read(client_key_path)
Simple enough so far, unfortunately, things begin to get a bit complicated. For starters, we previously only had to worry about a single signing algorithm: RSA + SHA256 (you might remember we refer to it as 'RS256'
in our header
method). We'll always use the SHA256 digest algorithm, regardless of the length of our RSA key.
With EC keys though, we'll use a different hashing algorithm depending on the curve used/key length (different curves = different key lengths):
Algorithm | Curve name | Key length (bits) | Hashing algorithm |
---|---|---|---|
ES256 | P-256 | 256 | SHA-256 |
ES384 | P-384 | 384 | SHA-384 |
ES512 | P-521 | 521 | SHA-512 |
Eagle-eyed readers might spot something odd about the ES512: we have a key 521 bits long, but the associated digest size is 512 bits. It's not a typo, and it does make things a bit more awkward; we can't assume that key size = digest size. We can get the key's bit length/curve using client_key.group.degree
, so let's write a method to get the associated digest size:
def digest_size
{ 256: 256, 384: 384, 521: 512 }[client_key.group.degree]
end
With this in place we can modify our hash_algo
method to dynamically fetch the correct Digest
class to match up with our EC key:
def hash_algo
OpenSSL::Digest.const_get("SHA#{digest_size}").new
end
Next, we need to modify our header
. We'll ditching be ditching our "e"
and "n"
keys (they're specific to RSA keys), and we'll need to add a "crv"
key to indicate the EC key's curve name. As we can see from the table above, for the curves we're concerned with, it's just the key's bit length prefixed with "P-"
. Lastly, alg
is now "ES"
+ digest_size
, and kty
(key type) is "EC"
:
@header ||= {
alg: "ES#{ digest_size }",
jwk: {
crv: "P-#{ client_key.group.degree }",
kty: 'EC'
}
}
Before we go any further, let's add a helper method which splits a string into pieces of a certain length (this will come in handy later):
def split_into_pieces(str, opts = {})
str.chars.each_slice(opts[:piece_size]).map(&:join)
end
# example:
split_into_pieces("abcdef", piece_size: 2)
# => ['ab', 'cd', 'ef']
Next, we have to add the public part of the key into the header. Running client_key.public_key.inspect
we see something like:
"#<OpenSSL::PKey::EC::Point:0x007fad4209d728 #...
The public part of an EC key is called a "public key curve point", and it's literally a point in 2-dimensional space. We need to provide the x
and y
coordinates of this point, again this is a little bit tricky. First, let's convert our public key to a hexidecimal string:
pub_key_hex = client_key.public_key.to_bn.to_s(16)
# => "04170BD2669BB4EA2DDFAD293F9B3F47703F671139F8C1FDE643ECC3B46DB519AA4BCAD1FB47566BC9C0730D5F6EE9C5FDA8D2DCF419F90C0BA6CFB669D80B80F9"
Andreas M. Antonopoulos, gives a good explanation of what we're looking at:
As we saw previously, the public key is a point on the elliptic curve consisting of a pair of coordinates (x,y). It is usually presented with the prefix
04
followed by two 256-bit numbers, one for the x coordinate of the point, the other for the y coordinate. The prefix04
is used to distinguish uncompressed public keys from compressed public keys that begin with a02
or a03
.
We don't really care about the 04
prefix - once we've got rid of that, we'll need to split our long hexidecimal sequence in half, to extract the x and y values.
First, let's use our split_into_pieces
to break it up into the individual octets:
pub_key_octets = split_into_pieces(pub_key_hex, piece_size: 2)
Next we'll drop our first octet (04
), and split our sequence in half:
pub_key_octets.shift # drop the first octet (which just indicates key is uncompressed)
x_octets, y_octets = pub_key_octets.each_slice(pub_key_octets.size / 2).to_a
Lastly, we'll convert our hex values to binary data using the pack method (see To Hex and Back (With Ruby) for a detailed explanation):
x = x_octets.map(&:hex).pack('c*')
y = y_octets.map(&:hex).pack('c*')
We can shorten our code a bit by converting to binary first, and reusing our split_into_pieces
method:
coords_binary_data = pub_key_octets.map(&:hex).pack('c*')
x, y = split_into_pieces(coords_binary_data, piece_size: coords_binary_data.size / 2)
Lastly, we'll need to Base64 encode our x
and y
values before sending them over the wire:
jwk: {
crv: "P-#{ client_key.group.degree }",
x: base64_le(x),
kty: 'EC',
y: base64_le(y)
}
To recap, our header
method now looks like this:
def header
@header ||= begin
pub_key_hex = client_key.public_key.to_bn.to_s(16)
pub_key_octets = split_into_pieces(pub_key_hex, piece_size: 2)
pub_key_octets.shift # drop the first octet (which just indicates key is uncompressed)
coords_binary_data = pub_key_octets.map(&:hex).pack('c*')
x, y = split_into_pieces(coords_binary_data, piece_size: coords_binary_data.length / 2)
{
alg: "ES#{ digest_size }",
jwk: {
crv: "P-#{ client_key.group.degree }",
x: base64_le(x),
kty: 'EC',
y: base64_le(y)
}
}
end
end
Worn out yet π
? There's one last step: we have to update how our signature is generated. When we sign a value using RSA, the signature is a single value Ο
, which is really just one long integer (see Digital signature on Wikipedia). But DSA (which we use with EC keys) returns a pair of integers, typically denoted r
and s
(see Wikipedia's DSA article), so we'll need to make a few modifications to allow for this. First, OpenSSL::PKey::EC
equivalent method to #sign
is #dsa_sign_asn1
:
signature = client_key.dsa_sign_asn1 hash_algo.digest([request[:protected], request[:payload]].join('.'))
Then we need to extract the value of (r
, s
) as binary strings. The signature is ASN.1 encoded, so we'll first decode it and convert it to an array (of two elements, i.e. r
and s
):
decoded_signature = OpenSSL::ASN1.decode(signature).to_a
Then we'll map the values of r
and s
as binary strings:
r, s = decoded_signature.map { |v| v.value.to_s(2) }
Finally, we set the "signature"
field in our JSON request to r
and s
concatenated together, and Base64 encoded:
request[:signature] = base64_le(r + s)
All the changes we needed to make are collected below (also see ec_client.rb
):
def client_key
@client_key ||= begin
client_key_path = File.expand_path('~/Desktop/ec-private.pem')
OpenSSL::PKey::EC.new IO.read(client_key_path)
end
end
def split_into_pieces(str, opts = {})
str.chars.each_slice(opts[:piece_size]).map(&:join)
end
def header
@header ||= begin
combined_coordinates = client_key.public_key.to_bn.to_s(16)
coord_octets = split_into_pieces(combined_coordinates, piece_size: 2)
coord_octets.shift # drop the first octet (which just indicates key is uncompressed)
coords_bin = coord_octets.map(&:hex).pack('c*')
x, y = split_into_pieces(coords_bin, piece_size: coords_bin.length / 2)
{
alg: "ES#{ client_key.group.degree }",
jwk: {
crv: "P-#{ client_key.group.degree }",
x: base64_le(x),
kty: 'EC',
y: base64_le(y)
}
}
end
end
def hash_algo
bit_size = client_key.group.degree
bit_size = 512 if bit_size == 521
OpenSSL::Digest.const_get("SHA#{bit_size}").new
end
def signed_request(url, payload)
request = {
payload: base64_le(payload),
header: header,
protected: base64_le(header.merge(nonce: nonce))
}
signature = client_key.dsa_sign_asn1 hash_algo.digest([request[:protected], request[:payload]].join('.'))
decoded_signature = OpenSSL::ASN1.decode(signature).to_a
r, s = decoded_signature.map { |v| v.value.to_s(2) }
request[:signature] = base64_le(r + s)
HTTParty.post(url, body: JSON.dump(request))
end
A fun factoid: Let's Encrypt certificates are technically only valid of 89 days and 23 hours, not for a whole 90 days. This is because LE backdates certificates by 1 hour to ensure the certificates can be validated immediately by clients whose clocks might be slightly out. Therefore a certificate issued on August 1st 12:34 will expire October 30th 11:34.
The validity period for Lets Encrypt certificates are relatively short. Per the CA/Browser Forum Baseline Requirements, Section 6.3.2:
Subscriber Certificates issued after 1 March 2018 MUST have a Validity Period no greater than 825 days. Subscriber Certificates issued after 1 July 2016 but prior to 1 March 2018 MUST have a Validity Period no greater than 39 months.
Accordingly, most commercial providers offer certificates with 1, 2 or 3 year validity periods (see GlobalSign's article on Maximum Certificate Validity). LE states the primary reasons for the shorter lifetime are:
- Shorter lifetimes decrease the compromise window in situations like Heartbleed
- Offering free certificates with a shorter lifetime provides encouragement for operators to automate issuance.
- Let's Encrypt's total capacity is bound by its OCSP signing capacity, and LE is required to sign OCSP responses for each certificate until it expires. Shorter expiry period means less overhead for certificates that were issued and then discarded, which in turn means higher total issuance capacity.
(Source: Pros and cons of 90-day certificate lifetimes)
Let's Encrypt will send email reminders to the address(es) provided in the contacts
field of your new-reg
payload, at the following times:
- 20 days before the date of expiry
- 10 days before the date of expiry
- 1 day before the expiry.
Additionally, various tools exist to monitor your certificates and alert you about upcoming expiries, including hosted services like LetsMonitor and Keychest or standalone applications like certinel. Dan Cvrcek posted a fairly extensive list on the LE forums.
If the private keys of our certificates get compromised, we need to disable certificates before they expire. In these cases we can explicitly revoke certificates; as the diagram above shows, to do this we make a signed request to LE which includes the certificate to be revoked. LE then propagates the revocation to certificate revocation lists and OCSP responders, which in turns should ensure browsers won't accept requests signed by the revoked certificate (especially if OCSP stapling is enabled, see Appendix 1).
There are a number of different ways to perform a revocation, depending on which keys you have access to.
Revocation requests are different from other ACME request in that they can be signed either with an account key pair or the key pair in the certificate. If we still have access to this key, we can simply load it in as our client key:
client_key_path = File.expand_path('~/Desktop/domain.key')
OpenSSL::PKey::RSA.new IO.read(client_key_path)
For our payload, we'll need the certificate in question. Since we have our private key locally, we'll assume the certificate is locally available too (though see Scenario 2 for alternative approaches):
cert_path = File.expand_path('~/Desktop/chained.pem')
certificate = OpenSSL::X509::Certificate.new IO.read(cert_path)
To revoke our certificate, we'll need to send a Base64 encoded version of the certificate in DER format:
new_registration = signed_request(endpoints['new-reg'], {
resource: 'revoke-cert',
certificate: base64_le(certificate.to_der)
})
Scenario 2: You don't have access to the private key for the certificate, but you still have access to the client key for the account which issued the certificate
If the authorizations are still valid for the certificate's domain (i.e. the certificate is less that 30 days old, as of April 2017), you can revoke the certificate as above, but using your existing account key:
client_key_path = File.expand_path('~/.ssh/id_rsa')
# ...
new_registration = signed_request(endpoints['new-reg'], {
resource: 'revoke-cert',
certificate: base64_le(certificate.to_der)
})
Note that you still need to provide the certificate in DER format, even if you're not providing the certificate's corresponding private key. You can dynamically fetch the certificate like so:
uri, certificate = URI.parse("https://example.com"), nil
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.start { |h| certificate = h.peer_cert }
Note that if it's been 30 days since you issued the certificate, the account key won't help you, and you're in an equivalent position to Scenario 3.
Scenario 3: You don't have access to the client key for the account which issued the certificate, or the private key for the domain, but you still control the certificate's domain(s)
Per LetsEncrypt's article on revocation:
If someone issued a certificate after compromising your host or your DNS, youβll want to revoke that certificate once you regain control. In order to revoke the certificate, Letβs Encrypt will need to ensure that you control the domain names in that certificate (otherwise people could revoke each otherβs certificates without permission)! To validate this control, Letβs Encrypt uses the same methods it uses to validate control for issuance: you can put a value in a DNS TXT record, put a file on an HTTP server, or offer a special TLS certificate.
In other words, you'll need to create a new account, pass the challenges for the domain(s) of the compromised certificate (see Section 4), then revoke the certificate as in Scenario 2, but using the account key for your newly created and authorized account.
- Bulletproof SSL and TLS - wonderful ~500 page book, goes into great detail about everything you might want to know about SSL/TLS
- SSL/TLS Deployment Best Practices (PDF) - By the same author as Bulletproof, a quick 10 page checklist
- TLS chapter in High Performance Browser Networking - like the TL;DR of Bulletproof, covers all the fundamentals, plus more recent developments like OCSP stapling, HSTS etc.
- OWASP's TLS Cheat Sheet - An excellent list of do's and don't relating to SSL/TLS
- Modern SSL/TLS Best Practices for Fast, Secure Websites (PDF, registration required) - decent white paper, with loads of visuals, up-to-date best practice recommendations
- SSL Best Practices: a Quick and Dirty Guide - top-level, reasonably recent (2015) best practices guide.
- How does SSL/TLS work? - Good StackExchange answer
- TLS in HTTP/2
- ACME spec - technical spec of the ACME protocol (which LE is built on)
- acme-tiny - a < 200 line Python client that served as the inspiration for this guide. The source code is v. readable + well commented
- Let's Encrypt - How It Works - official LE article
- Let's Encrypt Overview - good detailed article written back in 2015 when LE was starting out
- Let's Encrypt - A Certificate Authority To Encrypt the Entire Web (video)
- Using Free SSL/TLS Certificates from Let's Encrypt with NGINX
- Pros and cons of 90-day certificate lifetimes
Alex Peattie / alexpeattie.com / @alexpeattie
- Add Appendix 7 explaining how to use EC client keys
- Add Appendix 8 about certificate expiry and revocation
- Add note about terms of service URL now being available via the directory
- Update Appendix 4 with up-to-date rate limits, note about forthcoming wildcard certs
- Use the directory and response headers, rather than hardcoding URIs (closes #1)
- Add Appendix 6 about newly supported Internationalized Domain Names
- Change reference to official Let's Encrypt client β certbot
- Specify a TTL for DNS challenge record
- Add note about certificate and authorization validity periods
- Consistently prefer single quotes in all Ruby code
- Remove example domains for the various certificate types
- Added a couple more tools to the Testing section
- Add Changelog & Author section
- Harden example nginx config with additional security headers
- Initial release