Add deno.Buffer

[ry]

Do not confuse this with Node's Buffer. This is a direct port of Go's
bytes.ByteBuffer - it allows buffering of Reader objects.

[kitsonk]

Async generators are a thing and while not part of the GoLang API, implementing a [Symbol.asnycIterator]() would allow:

async function f() {
  for await (const x of denoBuffer) {
     console.log(x);
  }
}

Which relates to my other comments about having a more JavaScripty API.

[ry]

@kitsonk Regarding async iterators, there was a lengthy discussion a few months ago about whether Reader interfaces should be just async iterators: #387 (comment)

My conclusion was that it was overloading concerns and it would be better to provide a utility function for transforming any Reader into an async iterator.

function readerIterator(r: deno.Reader): AsyncIterator<ArrayBufferView>;

In particular, it's imperative that users are able to provide memory for the read() rather than having it allocated for them.

[GrosSacASac]

Hi, I am not a go expert, what is the difference with ArrayBuffer ?
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer

[kitsonk]

My only feedback would be toAsyncIterator(r: deno.Reader)... and I believe it would need to return a modified AsyncIterator so it can be cleanly used in for await in. Something like:

interface IterableAsyncIterator<T> extends AsyncIterator<T> {
  [Symbol.asyncIterator]: AsyncIterator<T>;
}

Or just an object with the Symbol.asyncIterator symbol on it and nothing else.

[kitsonk]

Hi, I am not a go expert, what is the difference with ArrayBuffer ?
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer

Functionally it conforms to the Deno's Reader/Writer APIs, so it can be easier to utilise with things like files and other readable/writable objects. The ArrayBuffer API does not conform, it is a more lower level construct. It does utilise a typed array within its implementation. The Web APIs don't really have good usable APIs around things like file reading and writing. So personally it being in the deno namespace makes it clear that it is something that Deno is adding to the mix. All the ArrayBuffer and associated APIs are still supported within Deno.

[bartlomieju]

@ry @kitsonk Regarding that users should be able to provide memory, how can we handle that with for-await-in?

Is there a way to pass ArrayBufferWay to async iterator in this example?

for await (const buf of toAsyncIterator(someReader)) {
   console.log(x);
}

[ry]

@GrosSacASac This is a Buffer that can grow. You can read() and write() from it. One situation where you might want to do this, is if you wanted to, say, buffer up an http response so it can be parsed:

import { dial, Buffer, copy, stdout } from "deno";
dial("tcp", "google.com:80").then((conn) => {
  let buf = new Buffer();
  buf.writeString("GET / HTTP/1.1\r\n\r\n");
  copy(conn, buf);
  copy(stdout, conn);
});

[ry]

@bartlomieju Ideally in the async iterator case we would allocate in Rust and pass it over to V8... This isn't implemented tho. So just allocate let buf = new Uint8Array(1024) and use that to pass to read().

[piscisaureus]

I really have trouble following what's going on here...

[piscisaureus]

I am not so sure how this is going to be used and whether it belongs in Deno.
But ok let's see where it goes.