RangeError: Offset is outside the bounds of the DataView (sparse file live streaming)

[lukepolo]

We have some code that throws an error when we are reading live data from memory using the inspect function:

const fileReader = fs.createReadStream(this.file, {
  fd: this.fd,
  autoClose: false,
  highWaterMark: this.chunkSize,
});

fileReader.on("data", (chunk: Buffer) => {
  try {
    return muxjs.mp4.tools.inspect(chunk);
  } catch (error) {
 
    logger.error("unable to read chunk", error);
    reject(error);
  }
});

fileReader.on("end", () => {
  resolve();
});

fileReader.on("error", (error) => {
  logger.error("Unable to read file properly", error);
});

RangeError: Offset is outside the bounds of the DataView
    at DataView.getUint16 (<anonymous>)
    at avc1 (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:71:34)
    at inspectMp4 (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:600:7)
    at stsd (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:430:29)
    at inspectMp4 (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:600:7)
    at stbl (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:389:16)
    at inspectMp4 (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:600:7)
    at minf (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:269:16)
    at inspectMp4 (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:600:7)
    at mdia (/home/vagrant/code/stream/node_modules/mux.js/lib/tools/mp4-inspector.js:254:16)

I believe this occurs because we have have half a moov in the buffer and the other half of the moov won't be till next the next buffer. I'm a novice when it comes to video, let me know what other information may be helpful.

[UPDATE]:
I was able to fix the error by pausing the read , and re-reading the file at the file position again. which allowed me to get the data properly

[lukepolo]

Here is a sample
226596175-424.bin.zip

[nklhtv]

i am trying to do exactly the same thing. Does this library supports progressive parsing?

[lukepolo]

You can, but what you want todo is keep adding to the buffer till you get a valid type. We also could not use the built in functions for streamer etc.

I eventually just wrote our own custom parser, as we only needed to find mp4 fragments (ftyp / moov -> moof / dat)

[nklhtv]

how can i know when i get a valid type. i will probably need something else than mux.js for that pourpose?

[lukepolo]

Basically this happens when you are trying to read live data, and the data is half there (your in the middle of a atom) and its not complete.

When this error happens, its telling you, that your data is not complete. To make it complete. keep the chunk from the previous read , and add it to the next chunk. This should allow you to get your data.

The problem that we had is , you dont know how much data was read from the previous chunk. So we opted to create our own parser that would rip out what has been read successfully.

This allowed us to know how much has been read, and if we had half an atom we could recover.

[nklhtv]

got it! Is that parser open sourced?

[lukepolo]

Sorry its not, but hit me up at my email and im always willing to help.