Write fails when path contains backslash chars

[pschmitt]

Hi,

the following script fails:

import hvac
import os

client = hvac.Client(
    url=os.environ.get('VAULT_ADDR'),
    token=os.environ.get('VAULT_TOKEN'),
    verify=os.environ.get('VAULT_SKIP_VERIFY', False)
)

client.write(
    'keyring/https://some.url/sub/entry',
    foo='bar'
)

Output:

Traceback (most recent call last):
  File "hvac_test.py", line 12, in <module>
    foo='bar'
  File "/home/pschmitt/.local/share/virtualenvs/vault-keyring-import/lib/python2.7/site-packages/hvac/v1/__init__.py", line 45, in write
    response = self._put('/v1/{0}'.format(path), json=kwargs)
  File "/home/pschmitt/.local/share/virtualenvs/vault-keyring-import/lib/python2.7/site-packages/hvac/v1/__init__.py", line 576, in _put
    return self.__request('put', url, **kwargs)
  File "/home/pschmitt/.local/share/virtualenvs/vault-keyring-import/lib/python2.7/site-packages/hvac/v1/__init__.py", line 606, in __request
    raise exceptions.InvalidRequest(errors=errors)
hvac.exceptions.InvalidRequest: missing data fields

I don't see why this fails since the request kwargs are correctly set to {'json': {'foo': 'bar'}}

Any ideas?

EDIT:

vault write "keyring/https://some.url/sub/entry" notes=okay                                                                                                 
Success! Data written to: keyring/https://some.url/sub/entr

This works just fine

[TerryHowe]

I don't know, but I would guess the cli is doing something similar to urlencoding and urldecoding the key for you. You would need to do that. Having a key to a REST API, you'd have to.

[ianunruh]

So, I'm able to reproduce this on my end, but I'm not sure how to fix it. In the native Vault client, there's no special escaping done on paths.

https://github.com/hashicorp/vault/blob/v0.5.2/api/logical.go#L47

Even when I escape that part of the URL with urllib.quote or urllib.quote_plus, I still get the missing data fields error.

[v6]

// , @ianunruh, are you saying that this could be an issue with the native Vault client?

[ianunruh]

The first thing that comes to mind is that it's a difference of how requests and the native Vault client are constructing the HTTP request. I'm not familiar enough with the internals of either library to be sure, however.

[ahlinc]

As I found due to double slash in request (...https://some...) we received 301 permanent redirect from the Vault server with location /v1/secret/http:/some.url/sub/entry and without double slash. On the response the Vault client makes second PUT request with new location and {"foo": "bar"} in the body but the requests library loses the body on second request. The resolve_redirects function looks a bit curiously and massive, so I suggest to add allow_redirects=False in the __request function and process all redirects on hvac library side accordingly to the standard Vault client.

[ianunruh]

Thanks to @ahlinc, I was able to fix this in 4db3276

The new behavior is to handle all redirects within HVAC, rather than in requests. If you wish to turn off this functionality, you can just pass allow_redirects=False to the client's constructor.

[thatarchguy]

Note: for Python 2.7.6 you have to specify allow_redirects=True or you will get a cryptic error. Functionality works fine in 2.7.11 and 3.5.1.

Ran into this error today and had to figure out why we couldn't read from vault.

>>> vault_server = "[vault url]"
>>> vault_token = "[vault_token]"
>>> client = hvac.Client(url=vault_server, token=vault_token, verify=False)
>>> client.read("some/key/here")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/case/env/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 38, in read
    return self._get('/v1/{0}'.format(path)).json()
  File "/opt/case/env/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 592, in _get
    return self.__request('get', url, **kwargs)
  File "/opt/case/env/local/lib/python2.7/site-packages/hvac/v1/__init__.py", line 619, in __request
    while response.is_redirect and self.allow_redirects:
AttributeError: 'Response' object has no attribute 'is_redirect'


>>> vault_server = "[vault url]"
>>> vault_token = "[vault_token]"
>>> client = hvac.Client(url=vault_server, token=vault_token, verify=False, allow_redirects=True)
>>> client.read("some/key/here")
/home/nuke/dev/env/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
{'auth': None,
 'data': {'host': 'host.test.com',
  'password': 'password',
  'username': 'usertest'},
 'lease_duration': 259200000,
 'lease_id': '',
 'renewable': False,
 'warnings': None}

[ianunruh]

@thatarchguy is_redirect was added in Requests 2.3.0. Which version of requests are you using?