401 Unauthorized Error

[FlogramMatt]

I've managed to get faster than requests running on Centos after compiling it myself, but when I post the same data to the same url using the same method as I use for python's normal request library.. I get an unauthorized error with this one.

using requests:

self._old_headers = {'content-type': 'application/json'}
self._session = requests.Session()
response = self._session.post(self._url, headers=self._old_headers, data=payload)#, verify=False)

using faster than requests:

self._fast_headers = [('content-type', 'application/json')]
response = fast_requests.post(url=self._url, http_headers=self._fast_headers, body=payload)

Unfortunately for security and confidentially reasons I can't share the url or payload.
But URL is in the form:
https://:@:

Payload is a number of json key-value pairs similar to:
{"method": "", "params": [], "": ""}

It appears this error for this application 401 Unauthorized Error should only be thrown if the username or password are incorrect, but I'm posting to the exact same url in both cases. Do I need to handle things differently for faster than requests?

Edit: The specific error I get returned minus a few hidden fields:
['', 'text/html; charset=ISO-8859-1', '401 Unauthorized', '1.1', 'https://:@:', '0', '{"content-length": @["0"], "content-type": @["text/html; charset=ISO-8859-1"], "date": @["Thu, 10 Jun 2021 16:50:59 GMT"], "www-authenticate": @["Basic realm=\"\""]}']

@juancarlospaco And Juan, I LOVE this library, so don't take this as a compliant, but might be nice to have a short guide that explains how to translate from 'requests' to 'faster-than-requests'.

[juancarlospaco]

Please paste full output of faster_than_requests.debugs()

[FlogramMatt]

faster-than-requests debugs():
{
"nimVersion": "1.4.4",
"httpCore": "Nim httpclient/1.4.4",
"cpu": "amd64",
"os": "linux",
"endian": "littleEndian",
"release": true,
"danger": true,
"CompileDate": "2021-06-01",
"CompileTime": "22:40:59",
"tempDir": "/tmp/",
"ssl": true,
"currentCompilerExe": "/home/centos/mczarnek/tools/nim-1.4.4/bin/nim",
"int.high": 9223372036854775807
}

[juancarlospaco]

On your request the Auth info goes into the Body ? or into the Header ?.
So if you try the same request using Postman or Insomnia does it work?.

The latest version of the compiler is 1.4.8 BTW.

[FlogramMatt]

This probably holds the answer for debugging this, the tcpdumps of each.

Looks like it's being added to the header if I'm reading these right?

Something that looks very interesting is that yours does not contain the "Authorizations" tag that requests does. (I cut out a few lines to make sure I'm not giving away a password)

0x00f0: 6774 683a 2035 390d 0a41 7574 686f 7269 gth:.59..Authori
0x0100: 7a61 7469 6f6e 3a20 4261 7369 6320 596d zation:.Basic.DF
0x0110: 6c30 5932 3970 626e 4a77 597a 6f78 4d6a lA9ca9xnJwYzoxMj
0x0120: 4e6e 5a6d 7072 5958 4e6b 5a6d 467a 5a47 NnZmprYXNkZmFzZG
0x0130: 5a68 6332 526d 4d31 497a 6345 4e44 566d RmMjMzc2E=....

REQUESTS (which works)
REQUESTS

sudo tcpdump -X -i ens192 host 10.10.0.18
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
15:18:38.630613 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [S], seq 4197395480, win 29200, options [mss 1460,sackOK,TS val 2087193654 ecr 0,nop,wscale 7], length 0
0x0000: 4500 003c 202e 4000 4006 4e1a c0a8 0171 E..<..@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2c18 0000 0000 ...Q...../,.....
0x0020: a002 7210 cca2 0000 0204 05b4 0402 080a ..r.............
0x0030: 7c68 0c36 0000 0000 0103 0307 |h.6........
15:18:38.631196 IP 10.10.0.18.8332 > localhost.localdomain.37122: Flags [S.], seq 951849523, ack 4197395481, win 28960, options [mss 1460,sackOK,TS val 160529118 ecr 2087193654,nop,wscale 7], length 0
0x0000: 4500 003c 0000 4000 3f06 6f48 0a0a 0051 E..<..@.?.oH...Q
0x0010: c0a8 0171 208c 9102 38bc 1233 fa2f 2c19 ...q....8..3./,.
0x0020: a012 7120 da86 0000 0204 05b4 0402 080a ..q.............
0x0030: 0991 7ade 7c68 0c36 0103 0307 ..z.|h.6....
15:18:38.631227 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [.], ack 1, win 229, options [nop,nop,TS val 2087193654 ecr 160529118], length 0
0x0000: 4500 0034 202f 4000 4006 4e21 c0a8 0171 E..4./@[email protected]!...q
0x0010: 0a0a 0051 9102 208c fa2f 2c19 38bc 1234 ...Q...../,.8..4
0x0020: 8010 00e5 cc9a 0000 0101 080a 7c68 0c36 ............|h.6
0x0030: 0991 7ade ..z.
15:18:38.631301 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [P.], seq 1:299, ack 1, win 229, options [nop,nop,TS val 2087193654 ecr 160529118], length 298
0x0000: 4500 015e 2030 4000 4006 4cf6 c0a8 0171 E..^.0@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2c19 38bc 1234 ...Q...../,.8..4
0x0020: 8018 00e5 cdc4 0000 0101 080a 7c68 0c36 ............|h.6
0x0030: 0991 7ade 504f 5354 202f 2048 5454 502f ..z.POST./.HTTP/
0x0040: 312e 310d 0a48 6f73 743a 2031 302e 3130 1.1..Host:.10.10
0x0050: 2e30 2e38 313a 3833 3332 0d0a 5573 6572 .0.81:8332..User
0x0060: 2d41 6765 6e74 3a20 7079 7468 6f6e 2d72 -Agent:.python-r
0x0070: 6571 7565 7374 732f 322e 3235 2e31 0d0a equests/2.25.1..
0x0080: 4163 6365 7074 2d45 6e63 6f64 696e 673a Accept-Encoding:
0x0090: 2067 7a69 702c 2064 6566 6c61 7465 0d0a .gzip,.deflate..
0x00a0: 4163 6365 7074 3a20 2a2f 2a0d 0a43 6f6e Accept:./..Con
0x00b0: 6e65 6374 696f 6e3a 206b 6565 702d 616c nection:.keep-al
0x00c0: 6976 650d 0a63 6f6e 7465 6e74 2d74 7970 ive..content-typ
0x00d0: 653a 2061 7070 6c69 6361 7469 6f6e 2f6a e:.application/j
0x00e0: 736f 6e0d 0a43 6f6e 7465 6e74 2d4c 656e son..Content-Len
0x00f0: 6774 683a 2035 390d 0a41 7574 686f 7269 gth:.59..Authori
0x0100: 7a61 7469 6f6e 3a20 4261 7369 6320 596d zation:.Basic.Df
0x0110: 6c30 5932 3970 626e 4a77 597a 6f78 4d6a lA9ca9xnJwYzoxMj
0x0120: 4e6e 5a6d 7072 5958 4e6b 5a6d 467a 5a47 NnZmprYXNkZmFzZG
0x0150: 526d 4d6a 4d7a 6332 453d 0d0a 0d0a RmMjMzc2E=....
15:18:38.631340 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [P.], seq 299:358, ack 1, win 229, options [nop,nop,TS val 2087193654 ecr 160529118], length 59
0x0000: 4500 006f 2031 4000 4006 4de4 c0a8 0171 E..o.1@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2d43 38bc 1234 ...Q...../-C8..4
0x0020: 8018 00e5 ccd5 0000 0101 080a 7c68 0c36 ............|h.6
0x0030: 0991 7ade 7b22 6d65 7468 6f64 223a 2022 ..z.{"method":."
0x0040: 6765 7462 6c6f 636b 636f 756e 7422 2c20 {censored}",.
0x0050: 2270 6172 616d 7322 3a20 5b5d 2c20 226a "params":.[],."j
0x0060: 736f 6e72 7063 223a 2022 322e 3022 7d sonrpc":."2.0"}
15:18:38.631697 IP 10.10.0.18.8332 > localhost.localdomain.37122: Flags [.], ack 299, win 235, options [nop,nop,TS val 160529119 ecr 2087193654], length 0
0x0000: 4500 0034 68eb 4000 3f06 0665 0a0a 0051 E..4h.@.?..e...Q
0x0010: c0a8 0171 208c 9102 38bc 1234 fa2f 2d43 ...q....8..4./-C
0x0020: 8010 00eb 785d 0000 0101 080a 0991 7adf ....x]........z.
0x0030: 7c68 0c36 |h.6
15:18:38.632409 IP 10.10.0.18.8332 > localhost.localdomain.37122: Flags [.], ack 358, win 235, options [nop,nop,TS val 160529119 ecr 2087193654], length 0
0x0000: 4500 0034 68ec 4000 3f06 0664 0a0a 0051 E..4h.@.?..d...Q
0x0010: c0a8 0171 208c 9102 38bc 1234 fa2f 2d7e ...q....8..4./-~
0x0020: 8010 00eb 7822 0000 0101 080a 0991 7adf ....x"........z.
0x0030: 7c68 0c36 |h.6
15:18:38.632429 IP 10.10.0.18.8332 > localhost.localdomain.37122: Flags [P.], seq 1:150, ack 358, win 235, options [nop,nop,TS val 160529120 ecr 2087193654], length 149
0x0000: 4500 00c9 68ed 4000 3f06 05ce 0a0a 0051 E...h.@.?......Q
0x0010: c0a8 0171 208c 9102 38bc 1234 fa2f 2d7e ...q....8..4./-~
0x0020: 8018 00eb cb9c 0000 0101 080a 0991 7ae0 ..............z.
0x0030: 7c68 0c36 4854 5450 2f31 2e31 2032 3030 |h.6HTTP/1.1.200
0x0040: 204f 4b0d 0a43 6f6e 7465 6e74 2d54 7970 .OK..Content-Typ
0x0050: 653a 2061 7070 6c69 6361 7469 6f6e 2f6a e:.application/j
0x0060: 736f 6e0d 0a44 6174 653a 2054 6875 2c20 son..Date:.Thu,.
0x0070: 3130 204a 756e 2032 3032 3120 3139 3a31 10.Jun.2021.19:1
0x0080: 383a 3338 2047 4d54 0d0a 436f 6e74 656e 8:38.GMT..Conten
0x0090: 742d 4c65 6e67 7468 3a20 3431 0d0a 0d0a t-Length:.41....
0x00a0: 7b22 7265 7375 6c74 223a 3638 3730 3930 {"result":687090
0x00b0: 2c22 6572 726f 7222 3a6e 756c 6c2c 2269 ,"error":null,"i
0x00c0: 6422 3a6e 756c 6c7d 0a d":null}.
15:18:38.632452 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [.], ack 150, win 237, options [nop,nop,TS val 2087193656 ecr 160529120], length 0
0x0000: 4500 0034 2032 4000 4006 4e1e c0a8 0171 E..4.2@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2d7e 38bc 12c9 ...Q...../-~8...
0x0020: 8010 00ed cc9a 0000 0101 080a 7c68 0c38 ............|h.8
0x0030: 0991 7ae0 ..z.
15:18:39.137250 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [F.], seq 358, ack 150, win 237, options [nop,nop,TS val 2087194160 ecr 160529120], length 0
0x0000: 4500 0034 2033 4000 4006 4e1d c0a8 0171 E..4.3@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2d7e 38bc 12c9 ...Q...../-~8...
0x0020: 8011 00ed cc9a 0000 0101 080a 7c68 0e30 ............|h.0
0x0030: 0991 7ae0 ..z.
15:18:39.137912 IP 10.10.0.18.8332 > localhost.localdomain.37122: Flags [F.], seq 150, ack 359, win 235, options [nop,nop,TS val 160529625 ecr 2087194160], length 0
0x0000: 4500 0034 68ee 4000 3f06 0662 0a0a 0051 E..4h.@.?..b...Q
0x0010: c0a8 0171 208c 9102 38bc 12c9 fa2f 2d7f ...q....8..../-.
0x0020: 8011 00eb 7397 0000 0101 080a 0991 7cd9 ....s.........|.
0x0030: 7c68 0e30 |h.0
15:18:39.137943 IP localhost.localdomain.37122 > 10.10.0.18.8332: Flags [.], ack 151, win 237, options [nop,nop,TS val 2087194161 ecr 160529625], length 0
0x0000: 4500 0034 2034 4000 4006 4e1c c0a8 0171 E..4.4@[email protected]....q
0x0010: 0a0a 0051 9102 208c fa2f 2d7f 38bc 12ca ...Q...../-.8...
0x0020: 8010 00ed cc9a 0000 0101 080a 7c68 0e31 ............|h.1
0x0030: 0991 7cd9 ..|.

FASTER-THAN-REQUESTS

sudo tcpdump -X -i ens192 host 10.10.0.18
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
15:17:24.688978 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [S], seq 251164874, win 29200, options [mss 1460,sackOK,TS val 2087119712 ecr 0,nop,wscale 7], length 0
0x0000: 4500 003c 93aa 4000 4006 da9d c0a8 0171 E..<..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 78ca 0000 0000 ...Q......x.....
0x0020: a002 7210 cca2 0000 0204 05b4 0402 080a ..r.............
0x0030: 7c66 eb60 0000 0000 0103 0307 |f......... 15:17:24.689471 IP 10.10.0.18.8332 > localhost.localdomain.36870: Flags [S.], seq 506535927, ack 251164875, win 28960, options [mss 1460,sackOK,TS val 160455177 ecr 2087119712,nop,wscale 7], length 0 0x0000: 4500 003c 0000 4000 3f06 6f48 0a0a 0051 E..<..@.?.oH...Q 0x0010: c0a8 0171 208c 9006 1e31 1ff7 0ef8 78cb ...q.....1....x. 0x0020: a012 7120 c87c 0000 0204 05b4 0402 080a ..q..|.......... 0x0030: 0990 5a09 7c66 eb60 0103 0307 ..Z.|f.....
15:17:24.689497 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [.], ack 1, win 229, options [nop,nop,TS val 2087119713 ecr 160455177], length 0
0x0000: 4500 0034 93ab 4000 4006 daa4 c0a8 0171 E..4..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 78cb 1e31 1ff8 ...Q......x..1..
0x0020: 8010 00e5 cc9a 0000 0101 080a 7c66 eb61 ............|f.a
0x0030: 0990 5a09 ..Z.
15:17:24.689559 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [P.], seq 1:153, ack 1, win 229, options [nop,nop,TS val 2087119713 ecr 160455177], length 152
0x0000: 4500 00cc 93ac 4000 4006 da0b c0a8 0171 E.....@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 78cb 1e31 1ff8 ...Q......x..1..
0x0020: 8018 00e5 cd32 0000 0101 080a 7c66 eb61 .....2......|f.a
0x0030: 0990 5a09 504f 5354 202f 2048 5454 502f ..Z.POST./.HTTP/
0x0040: 312e 310d 0a48 6f73 743a 2031 302e 3130 1.1..Host:.10.10
0x0050: 2e30 2e38 313a 3833 3332 0d0a 436f 6e6e .0.81:8332..Conn
0x0060: 6563 7469 6f6e 3a20 4b65 6570 2d41 6c69 ection:.Keep-Ali
0x0070: 7665 0d0a 636f 6e74 656e 742d 6c65 6e67 ve..content-leng
0x0080: 7468 3a20 3539 0d0a 636f 6e74 656e 742d th:.59..content-
0x0090: 7479 7065 3a20 6170 706c 6963 6174 696f type:.applicatio
0x00a0: 6e2f 6a73 6f6e 0d0a 7573 6572 2d61 6765 n/json..user-age
0x00b0: 6e74 3a20 4e69 6d20 6874 7470 636c 6965 nt:.Nim.httpclie
0x00c0: 6e74 2f31 2e34 2e34 0d0a 0d0a nt/1.4.4....
15:17:24.689949 IP 10.10.0.18.8332 > localhost.localdomain.36870: Flags [.], ack 153, win 235, options [nop,nop,TS val 160455177 ecr 2087119713], length 0
0x0000: 4500 0034 45f1 4000 3f06 295f 0a0a 0051 E..4E.@.?.)_...Q
0x0010: c0a8 0171 208c 9006 1e31 1ff8 0ef8 7963 ...q.....1....yc
0x0020: 8010 00eb 66e5 0000 0101 080a 0990 5a09 ....f.........Z.
0x0030: 7c66 eb61 |f.a
15:17:24.689965 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [P.], seq 153:212, ack 1, win 229, options [nop,nop,TS val 2087119713 ecr 160455177], length 59
0x0000: 4500 006f 93ad 4000 4006 da67 c0a8 0171 E..o..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 7963 1e31 1ff8 ...Q......yc.1..
0x0020: 8018 00e5 ccd5 0000 0101 080a 7c66 eb61 ............|f.a
0x0030: 0990 5a09 7b22 6d65 7468 6f64 223a 2022 ..Z.{"method":."
0x0040: 6765 7462 6c6f 636b 636f 756e 7422 2c20 {censored}",.
0x0050: 2270 6172 616d 7322 3a20 5b5d 2c20 226a "params":.[],."
0x0060: 736f 6e72 7063 223a 2022 322e 3022 7d {censored}":."2.0"}
15:17:24.690328 IP 10.10.0.18.8332 > localhost.localdomain.36870: Flags [.], ack 212, win 235, options [nop,nop,TS val 160455178 ecr 2087119713], length 0
0x0000: 4500 0034 45f2 4000 3f06 295e 0a0a 0051 E..4E.@.?.)^...Q
0x0010: c0a8 0171 208c 9006 1e31 1ff8 0ef8 799e ...q.....1....y.
0x0020: 8010 00eb 66a9 0000 0101 080a 0990 5a0a ....f.........Z.
0x0030: 7c66 eb61 |f.a
15:17:24.690951 IP 10.10.0.18.8332 > localhost.localdomain.36870: Flags [P.], seq 1:172, ack 212, win 235, options [nop,nop,TS val 160455178 ecr 2087119713], length 171
0x0000: 4500 00df 45f3 4000 3f06 28b2 0a0a 0051 E...E.@.?.(....Q
0x0010: c0a8 0171 208c 9006 1e31 1ff8 0ef8 799e ...q.....1....y.
0x0020: 8018 00eb 7522 0000 0101 080a 0990 5a0a ....u"........Z.
0x0030: 7c66 eb61 4854 5450 2f31 2e31 2034 3031 |f.aHTTP/1.1.401
0x0040: 2055 6e61 7574 686f 7269 7a65 640d 0a57 .Unauthorized..W
0x0050: 5757 2d41 7574 6865 6e74 6963 6174 653a WW-Authenticate:
0x0060: 2042 6173 6963 2072 6561 6c6d 3d22 6a73 .Basic.realm="
0x0070: 6f6e 7270 6322 0d0a 4461 7465 3a20 5468 {censored}"..Date:.Th
0x0080: 752c 2031 3020 4a75 6e20 3230 3231 2031 u,.10.Jun.2021.1
0x0090: 393a 3137 3a32 3420 474d 540d 0a43 6f6e 9:17:24.GMT..Con
0x00a0: 7465 6e74 2d4c 656e 6774 683a 2030 0d0a tent-Length:.0..
0x00b0: 436f 6e74 656e 742d 5479 7065 3a20 7465 Content-Type:.te
0x00c0: 7874 2f68 746d 6c3b 2063 6861 7273 6574 xt/html;.charset
0x00d0: 3d49 534f 2d38 3835 392d 310d 0a0d 0a =ISO-8859-1....
15:17:24.690970 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [.], ack 172, win 237, options [nop,nop,TS val 2087119714 ecr 160455178], length 0
0x0000: 4500 0034 93ae 4000 4006 daa1 c0a8 0171 E..4..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 799e 1e31 20a3 ...Q......y..1..
0x0020: 8010 00ed cc9a 0000 0101 080a 7c66 eb62 ............|f.b
0x0030: 0990 5a0a ..Z.
15:17:24.691024 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [F.], seq 212, ack 172, win 237, options [nop,nop,TS val 2087119714 ecr 160455178], length 0
0x0000: 4500 0034 93af 4000 4006 daa0 c0a8 0171 E..4..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 799e 1e31 20a3 ...Q......y..1..
0x0020: 8011 00ed cc9a 0000 0101 080a 7c66 eb62 ............|f.b
0x0030: 0990 5a0a ..Z.
15:17:24.691324 IP 10.10.0.18.8332 > localhost.localdomain.36870: Flags [F.], seq 172, ack 213, win 235, options [nop,nop,TS val 160455179 ecr 2087119714], length 0
0x0000: 4500 0034 45f4 4000 3f06 295c 0a0a 0051 E..4E.@.?.)...Q
0x0010: c0a8 0171 208c 9006 1e31 20a3 0ef8 799f ...q.....1....y.
0x0020: 8011 00eb 65fa 0000 0101 080a 0990 5a0b ....e.........Z.
0x0030: 7c66 eb62 |f.b
15:17:24.691342 IP localhost.localdomain.36870 > 10.10.0.18.8332: Flags [.], ack 173, win 237, options [nop,nop,TS val 2087119714 ecr 160455179], length 0
0x0000: 4500 0034 93b0 4000 4006 da9f c0a8 0171 E..4..@[email protected]
0x0010: 0a0a 0051 9006 208c 0ef8 799f 1e31 20a4 ...Q......y..1..
0x0020: 8010 00ed cc9a 0000 0101 080a 7c66 eb62 ............|f.b
0x0030: 0990 5a0b ..Z.

[FlogramMatt]

Can't make same request via postman or insomnia because it's on a server that I have to ssh into and only have command line access. However, I can make the request with curl and it works.

Working on recreating it and will send through tcpdump too

Also something I filtered out but probably relevant and not a big deal to share: it's using json-rpc

[juancarlospaco]

Correct, it does not auto-inject anything into the headers.

You have to pass headers explicitly, kinda like:

[("Content-Type", "application/json"), ("Authorization": "Basic user:pass")]

There "user" and "password" go Base64'ed. I did not invent this, this is just how it works.

[FlogramMatt]

Ok, I see. Figured it would take care of it like requests does, though this is made for speed and from that point of view, I'm glad it doesn't as it'd have to recalculate every single time.

Base64'd, good to know. Might be good to add that as a comment somewhere as I'm sure I won't be the last one to try to send a request that requires username/pass Thank you.

It is now working, thank you again Juan!