Can't upload binary files in Python2 because body is converted to unicode

[ghost]

• Python Version: 2.7.12
• Requests Version: 2.12.4
• Requests-Oauthlib version: 0.7.0

Appears somewhat related to this issue: https://github.com/kennethreitz/requests/issues/1252

We're trying to send a multipart/form-data request that includes a text field and a binary file. All our code uses regular strings but it appears that requests_oauthlib converts the message body to a unicode string and when httplib tries to append the binary data it gives the following error:

self = <requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x00000000072792E8>
message_body = '--cb837e5a8c104677b01e697c20039c41\r\nContent-Disposition: form-data; name="async"\r\n\r\nTrue\r\n--cb837e5a8c104677b...00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\r\n--cb837e5a8c104677b01e697c20039c41--\r\n'

    def _send_output(self, message_body=None):
        """Send the currently buffered request and clear the buffer.

            Appends an extra \\r\\n to the buffer.
            A message_body may be specified, to be appended to the request.
            """
        self._buffer.extend(("", ""))
        msg = "\r\n".join(self._buffer)
        del self._buffer[:]
        # If msg and message_body are sent in a single send() call,
        # it will avoid performance problems caused by the interaction
        # between delayed ack and the Nagle algorithm.
        if isinstance(message_body, str):
>           msg += message_body
E           UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 333: ordinal not in range(128)

..\..\appdata\local\continuum\miniconda2\envs\testenv\lib\httplib.py:895: UnicodeDecodeError

[Lukasa]

Hrm, this seems problematic. Is it possible for you to provide a minimal reproducer that uses httpbin.org?

[ghost]

I'm not sure how to use httpbin with oauth2, but upon further investigation this seems to happen specifically when using prepared requests, here's a snippet that reproduces the error

from io import BytesIO
from oauthlib.oauth2 import LegacyApplicationClient
from requests_oauthlib import OAuth2Session, OAuth2
from requests import Request, Session


username = 'myuser'
password = 'mypassword'
client_id = 'myclientid'
client_secret = 'myclientsecret'

client = LegacyApplicationClient(client_id=client_id)
session = OAuth2Session(client=client)
token = session.fetch_token(
    token_url='https://httpbin.org/post',
    username=username,
    password=password,
    client_id=client_id,
    client_secret=client_secret)
req = Request(
    'post',
    'https://httpbin.org/post',
    data={'name': 'TestFile'},
    files={'file': BytesIO('\x89')},
    auth=OAuth2(client=client))
prep = session.prepare_request(req)
res = session.send(prep)

[Lukasa]

So, I don't think the OAuth2Session supports the prepared request flow: you have to use the request method. Any reason you aren't doing that?

[ghost]

Been adapting an old project so I can't speak for why the original used prepared requests. Adding an OAuth2 instance to a regular requests library Session object causes the same issue though so it appears to be a problem with the OAuth2 class rather than the OAuth2Session class.

[singingwolfboy]

Hi @rainyday, it's been over a year since there was any activity on this issue. Is this still a problem for you? If not, I'm going to close the issue.

[asm128]

Hi @rainyday, it's been over a year since there was any activity on this issue. Is this still a problem for you? If not, I'm going to close the issue.

What would be the reason for closing this? Do you know at least if this is still reproducible?