Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(ext/headers): cache iterableHeaders for immutable Headers #20132

Merged
merged 1 commit into from
Aug 12, 2023

Conversation

marcosc90
Copy link
Contributor

@marcosc90 marcosc90 commented Aug 11, 2023

This PR caches _iterableHeaders for immutable Headers increasing the performance of fetch & server if headers are iterated.

Should close #19466

I only cached immutable headers to address this comment #19466 (comment) since I didn't find any occurrence of header mutation on immutable headers. We can discuss caching for non-immutable, but I think this is a great first step.

BENCHMARK

Server

const addr = Deno.args[0] ?? "127.0.0.1:4500";
const [hostname, port] = addr.split(":");
const { serve } = Deno;

serve({ hostname, port: Number(port), reusePort: true }, (req) => {
  const headers = [...req.headers]; // req.headers are immutable, cannot set/append/delete
  return new Response("ok");
});

Used wrk with 5 headers

wrk -d 10s --latency -H "X-Deno: true" -H "Accept: application/json" -H "X-Foo: bar" -H "User-Agent: wrk" -H "Accept-Encoding: gzip, br" http:https://127.0.0.1:4500

This patch

Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    70.18us   22.89us 679.00us   81.37%
    Req/Sec    71.55k     9.69k   82.18k    89.60%
  Latency Distribution
     50%   59.00us
     75%   89.00us
     90%   98.00us
     99%  159.00us
  1437891 requests in 10.10s, 193.35MB read
Requests/sec: 142369.83
Transfer/sec:     19.14MB

main

Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   112.78us   36.47us   2.09ms   77.99%
    Req/Sec    44.30k     1.65k   49.14k    74.26%
  Latency Distribution
     50%   99.00us
     75%  136.00us
     90%  162.00us
     99%  213.00us
  890588 requests in 10.10s, 118.91MB read
Requests/sec:  88176.37
Transfer/sec:     11.77MB

fetch

const res = await fetch('http:https://127.0.0.1:4500');
Deno.bench("Headers iterator", () => {
  const i = [...res.headers]; // res.headers are immutable, cannot set/append/delete
});

this patch

cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator      329.5 ns/iter   3,034,909.0 (318.55 ns … 364.34 ns)  331.1 ns 355.72 ns 364.34 ns

main

cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator       2.59 µs/iter     386,372.1     (2.56 µs … 2.68 µs)   2.59 µs   2.68 µs   2.68 µs

Copy link
Contributor

@mmastrac mmastrac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mmastrac mmastrac merged commit f843a1f into denoland:main Aug 12, 2023
13 checks passed
@marcosc90 marcosc90 deleted the perf-immutable-headers branch August 12, 2023 16:58
littledivy pushed a commit to littledivy/deno that referenced this pull request Aug 21, 2023
…and#20132)

This PR caches `_iterableHeaders` for immutable `Headers` increasing the
performance of `fetch` & server if headers are iterated.

Should close denoland#19466 

I only cached immutable headers to address this comment
denoland#19466 (comment)
since I didn't find any occurrence of header mutation on immutable
headers. We can discuss caching for non-immutable, but I think this is a
great first step.

## BENCHMARK

### Server
```js
const addr = Deno.args[0] ?? "127.0.0.1:4500";
const [hostname, port] = addr.split(":");
const { serve } = Deno;

serve({ hostname, port: Number(port), reusePort: true }, (req) => {
  const headers = [...req.headers]; // req.headers are immutable, cannot set/append/delete
  return new Response("ok");
});

```
Used `wrk` with 5 headers
```
wrk -d 10s --latency -H "X-Deno: true" -H "Accept: application/json" -H "X-Foo: bar" -H "User-Agent: wrk" -H "Accept-Encoding: gzip, br" http:https://127.0.0.1:4500
```


**This patch**
```
Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    70.18us   22.89us 679.00us   81.37%
    Req/Sec    71.55k     9.69k   82.18k    89.60%
  Latency Distribution
     50%   59.00us
     75%   89.00us
     90%   98.00us
     99%  159.00us
  1437891 requests in 10.10s, 193.35MB read
Requests/sec: 142369.83
Transfer/sec:     19.14MB
```
**main**
```
Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   112.78us   36.47us   2.09ms   77.99%
    Req/Sec    44.30k     1.65k   49.14k    74.26%
  Latency Distribution
     50%   99.00us
     75%  136.00us
     90%  162.00us
     99%  213.00us
  890588 requests in 10.10s, 118.91MB read
Requests/sec:  88176.37
Transfer/sec:     11.77MB
```
### fetch

```js
const res = await fetch('http:https://127.0.0.1:4500');
Deno.bench("Headers iterator", () => {
  const i = [...res.headers]; // res.headers are immutable, cannot set/append/delete
});
```

**this patch**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator      329.5 ns/iter   3,034,909.0 (318.55 ns … 364.34 ns)  331.1 ns 355.72 ns 364.34 ns
```
**main**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator       2.59 µs/iter     386,372.1     (2.56 µs … 2.68 µs)   2.59 µs   2.68 µs   2.68 µs
```
littledivy pushed a commit that referenced this pull request Aug 21, 2023
This PR caches `_iterableHeaders` for immutable `Headers` increasing the
performance of `fetch` & server if headers are iterated.

Should close #19466 

I only cached immutable headers to address this comment
#19466 (comment)
since I didn't find any occurrence of header mutation on immutable
headers. We can discuss caching for non-immutable, but I think this is a
great first step.

## BENCHMARK

### Server
```js
const addr = Deno.args[0] ?? "127.0.0.1:4500";
const [hostname, port] = addr.split(":");
const { serve } = Deno;

serve({ hostname, port: Number(port), reusePort: true }, (req) => {
  const headers = [...req.headers]; // req.headers are immutable, cannot set/append/delete
  return new Response("ok");
});

```
Used `wrk` with 5 headers
```
wrk -d 10s --latency -H "X-Deno: true" -H "Accept: application/json" -H "X-Foo: bar" -H "User-Agent: wrk" -H "Accept-Encoding: gzip, br" http:https://127.0.0.1:4500
```


**This patch**
```
Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    70.18us   22.89us 679.00us   81.37%
    Req/Sec    71.55k     9.69k   82.18k    89.60%
  Latency Distribution
     50%   59.00us
     75%   89.00us
     90%   98.00us
     99%  159.00us
  1437891 requests in 10.10s, 193.35MB read
Requests/sec: 142369.83
Transfer/sec:     19.14MB
```
**main**
```
Running 10s test @ http:https://127.0.0.1:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   112.78us   36.47us   2.09ms   77.99%
    Req/Sec    44.30k     1.65k   49.14k    74.26%
  Latency Distribution
     50%   99.00us
     75%  136.00us
     90%  162.00us
     99%  213.00us
  890588 requests in 10.10s, 118.91MB read
Requests/sec:  88176.37
Transfer/sec:     11.77MB
```
### fetch

```js
const res = await fetch('http:https://127.0.0.1:4500');
Deno.bench("Headers iterator", () => {
  const i = [...res.headers]; // res.headers are immutable, cannot set/append/delete
});
```

**this patch**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator      329.5 ns/iter   3,034,909.0 (318.55 ns … 364.34 ns)  331.1 ns 355.72 ns 364.34 ns
```
**main**
```
cpu: 13th Gen Intel(R) Core(TM) i9-13900H
runtime: deno 1.36.1 (x86_64-unknown-linux-gnu)

benchmark             time (avg)        iter/s             (min … max)       p75       p99      p995
---------------------------------------------------------------------- -----------------------------
Headers iterator       2.59 µs/iter     386,372.1     (2.56 µs … 2.68 µs)   2.59 µs   2.68 µs   2.68 µs
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Headers._iterableHeaders could use caching
2 participants