Support mTLS client certificate authenticating

[samutamm]

Is your feature request related to a problem? Please describe.
In client-server communication, where client is browser or other user terminal, using HTTPS/TLS helps client to trust the server, as server certificate is authenticated by 3rd party. In some scenarios (IOT, microservices, etc) it might be important for server to ensure that client is not compromised. Authenticating client certificates would help server to trust the client.

Describe the solution you'd like
Create new Mutual TLS (mTLS) feature. When enabled, Easegress verifies that TLS certificates of the connecting client are signed by same CA certificates as Easegress certificates. Client will equally verify certificates of Easegress.

Describe alternatives you've considered
There's no alternative way for Easegress server to trust the client.

Additional context

• https://en.wikipedia.org/wiki/Mutual_authentication
• https://en.wikipedia.org/wiki/Certificate_authority
• https://github.com/haoel/mTLS

[samutamm]

It turned out mTLS is already supported (#281).

HTTPServer enable mTLS by setting https to true and providing CA certificate in caCertBase64;

kind: HTTPServer
...
https: true
caCertBase64: "CA_CERT"
certs:
  sub.domain.io: "SERVER_CERT_CRT"
keys:
  sub.domain.io: "SERVER_CERT_KEY"

However there's still room for improvement. Currently updating certificates requires deleting and re-creating HTTPServer, causing service downtime. AutoCertManager helps with automatic certificate renewal, but for mTLS there are still use cases when certificates should be updated manually. For example, if certificates of one client are revoked and new certificates are created for the rest of the clients and server, there should be a way to update those new certificates to Easegress without restarting HTTPServer.

Design draft - HTTPServer certificateStorage

Here's an alternative proposal that synchronizes certifications from Easegress custom-data store.

kind: HTTPServer
...
https: true
# following entry is new
certificateStorage:
  prefix: "certificates/"
# caCertBase64, certs and keys are left empty

Certificates can be stored to Easegress using egctl:

echo '                                       
rebuild: true
list:
  - caCertBase64: 'CA_CERT'
     certCrtBase64: 'SERVER_CERT_CRT'
     certKeyBase64: 'SERVER_CERT_KEY'
' | egctl custom-data update certificates

Another option is to add this certificateStorage feature to AutoCertManager.

Design draft - AutoCertManager certificateStorage

AutoCertManager needs to only know, where to look for certificates.

kind: AutoCertManager
name: certificateStorage
certificateStorage:
  prefix: "certificates/"
# everything else left empty

And egctl custom-data can store certificates, like in previous case.

To avoid overloading this new feature to existing objects HTTPServer and AutoCertManager, one option is also to create new object for this. Let's say CertificateStorage.

Design draft - CertSynchronizer

This controller synchronize the certificates from Easegress custom data.

kind: CertSynchronizer
name: certSync
prefix: "certificates/"

Again, certs are stored with same egctl custom-data command.

@localvar @suchen-sci @zouyingjie What do you think? Do we need certificate synchronization or is it sufficient to re-create HTTPServer every time certificates change? If we need certificate synchronizer, which one the drafts makes most sense for you?

[localvar]

I prefer to update AutoCertManager to support both server & client certificates.

And for the original requirement, like the HTTP basic auth validator, we also need to put the client id into the header of the request.

[samutamm]

Thanks for input @localvar !

So currently it's possible to secure communication between clients and Easegress using mTLS, using fixed certificates. Updating mTLS requires re-creating HTTPServer. I plan to support updating mTLS certificates (CA cert and domain specific certs) using egctl custom data feature and AutoCertManager Easegress object. Before going in to the details, here's the big picture for updating mTLS certificates:

Implementation details

• HTTPServer spec.tlsConfig() already supports server (.crt & .key) certs from AutoCertManager and CA cert from HTTPServer spec. Modify it to fetch CA cert also from AutoCertManager.
• Support following AutoCertManager configs:

kind: AutoCertManager
name: certificateStorage
certificateStorage:
  prefix: "certificates/"
# everything else left empty

• When certificateStorage is given, AutoCertManager doesn't use ACME & Let's Encrypt but watches certificates in cluster storage using storage.watchCertificate
• Modify storage.watchCertificate and other functions to support both ACME and user managed certificates

For

And for the original requirement, like the HTTP basic auth validator, we also need to put the client id into the header of the request.

I don't know clean way for HTTPServer or AutoCertManager to update request headers. I think it would be better to use AutoCertManager for securing connections using mTLS and BasicAuthentication validator to authenticate and add authenticated client id into headers.

[localvar]

For

When certificateStorage is given, AutoCertManager doesn't use ACME & Let's Encrypt but watches certificates in cluster storage using storage.watchCertificate

I think the certificateStorage is only for CA certs (to verify clients), ACME should still be used for server-side certs.

And for

And for the original requirement, like the HTTP basic auth validator, we also need to put the client id into the header of the request.

Please check https://github.com/haoel/mTLS/blob/e7044955900f2d9c00e9b33f024380522caa983b/server.go#L46 , we can get the client certs from the request, that's we get the CN from the client cert and add the client id (need to convert CN to client ID) to the header.

Actually, the 2nd point is the major one for the original requirement (multi-tenant kafka), while the 1st one is nice to have.

[samutamm]

OK I see, let's support first parsing the client ID (or any information) from the client certificates. Let's leave updating certificates without re-creating HTTPServer (#467 (comment)) for later.

Here's a draft config for a Filter CertExtractor that extracts information from request certificate and adds it to headers:

kind: "CertSubjectExtractor"
name: "cn-extractor"
certIndex: -1 # n'th certificate; 0 for first, 1 for second, …, -2 for one before last, -1 for last
target: "subject" # subject or issuer
field: "CommonName" # country, province, orgranization.. etc
headerKey: "tls-subject-cn" # header key where to set extracted info