HVAC authentication fail with self-signed certificate but works with vault API/CLI

[juigilkishore]

Unable to authenticate vault with HVAC when TLS enabled with self-signed certificate

Started Vault service with TLS enabled

listener "tcp" {
 address     = "172.18.0.10:8200"
 tls_cert_file = "self-signed.crt"
 tls_key_file = "self-signed.key"
}

where the self-signed.crt and self-signed.key is self signed certificate-key pair generated through openssl

[user@centos hashicorp-vault]$ openssl x509 -noout -text -in self-signed.crt | grep -E -A1 'Subject:|Issuer:|X509v3'
        Issuer: C=IN, ST=KA, L=BLR, O=MY ORG, OU=MY ORG UNIT, CN=MY NODE
        Validity
--
        Subject: C=IN, ST=KA, L=BLR, O=MY ORG, OU=MY ORG UNIT, CN=MY NODE
        Subject Public Key Info:
--
        X509v3 extensions:
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Key Usage:
                Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
            X509v3 Subject Alternative Name:
                DNS:centos.domain.com, IP Address:172.18.0.10

Vault CLI/API works with self-signed certificate

Able to perform unseal, status check, read, write secrets to kv-v2 engine (CLI/API) with ca-cert flag

[user@centos hashicorp-vault]$ env |grep VAULT
VAULT_ADDR=https://172.18.0.10:8200/
VAULT_TOKEN=s.Kqk7fRyUT3yI9hR1PStitRkJ
[user@centos hashicorp-vault]$ vault kv get -format=json -field=data -ca-cert=self-signed.crt my-secret/info
{
  "key1": "val1",
  "key2": "val2"
}
[user@centos hashicorp-vault]$ curl -s https://172.18.0.10:8200/v1/my-secret/data/info -H "X-Vault-Token: s.Kqk7fRyUT3yI9hR1PStitRkJ" --cacert self-signed.crt | jq -r .data.data
{
  "key1": "val1",
  "key2": "val2"
}
[user@centos hashicorp-vault]$
[user@centos hashicorp-vault]$ curl -s https://172.18.0.10:8200/v1/auth/token/lookup-self -H "X-Vault-Token: s.Kqk7fRyUT3yI9hR1PStitRkJ" --cacert self-signed.crt  | jq -r .data.id
s.Kqk7fRyUT3yI9hR1PStitRkJ
[user@centos hashicorp-vault]$

HVAC authenticate fails with self-signed certificate

Using hvac 0.10.5 version and python 2.7. Tried the below three combinations (with verify flag only, cert flag only and both verify and cert flags). All of them resulted in the same SSL CERTIFICATE_VERIFY_FAILED Exception

import hvac

cacert = "self-signed.crt"
cert=("self-signed.crt", "self-signed.key")

client = hvac.Client(url="https://172.18.0.10:8200", token="s.Kqk7fRyUT3yI9hR1PStitRkJ", verify=cacert)
try: client.is_authenticated()
except Exception as e: print("E1", e)

client = hvac.Client(url="https://172.18.0.10:8200", token="s.Kqk7fRyUT3yI9hR1PStitRkJ", cert=cert)
try: client.is_authenticated()
except Exception as e: print("E2", e)

client = hvac.Client(url="https://172.18.0.10:8200", token="s.Kqk7fRyUT3yI9hR1PStitRkJ", verify=cacert, cert=cert)
try: client.is_authenticated()
except Exception as e: print("E3", e)

('E1', SSLError(MaxRetryError("HTTPSConnectionPool(host='172.18.0.10', port=8200): Max retries exceeded with url: /v1/auth/token/lookup-self (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)'),))",),))
('E2', SSLError(MaxRetryError("HTTPSConnectionPool(host='172.18.0.10', port=8200): Max retries exceeded with url: /v1/auth/token/lookup-self (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)'),))",),))
('E3', SSLError(MaxRetryError("HTTPSConnectionPool(host='172.18.0.10', port=8200): Max retries exceeded with url: /v1/auth/token/lookup-self (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)'),))",),))

1. Is something missed while initializing the hvac client ? If not the main concern is why the same certificate is working for API/CLI but not with python client ? (or)
2. Anything wrong with the self-signed certificate, does it need to be signed by a CA or have any specific X509v3 extensions ?

[jeffwecan]

You may need to point the underlying requests module to a CA bundle that includes the issuer certificate for your self-signed certificate. Perhaps you could try reading through https://hvac.readthedocs.io/en/stable/advanced_usage.html#making-use-of-private-ca and letting us know if that helps navigate the issue or if it doesn't we can explore the issue further and perhaps clarifiy this section of the documentation.

[Tylerlhess]

This might be solved with pr #691 . I added some extra logging around this with documentation that to authenticate with a non-standard CA you need to provide that CA as the verify kwarg. It also allows setting/changing the certs outside of a single statement client creation and login (which was the only way to use certificate auth previously)

client = hvac.Client(. details including the certs and ca ).auth_tls()

where as now you can create the client and then auth

client =hvac.Client
client.auth.cert.login(. certts and ca. )