Reading data files from an npm package shouldn't require allow-read permssions

[ry]

This npm module reads from a data file:

import {encode, decode} from 'npm:gpt-3-encoder';

const str = 'This is an example sentence to try encoding out on!'
const encoded = encode(str)
console.log('Encoded this string looks like: ', encoded)

console.log('We can look at each token and what it represents')
for(let token of encoded){
  console.log({token, string: decode([token])})
}

const decoded = decode(encoded)
console.log('We can decode it back into:\n', decoded)

Ideally this could run without any permissions.

[ghost]

Is anyone currently working on this? I was curious about how deno was loading NPM packages prior, correct me if I am wrong here about anything. The npm package utilizing fs,readFileSync to read ./encoder.json line 5.

https://www.npmjs.com/package/gpt-3-encoder?activeTab=code

The npm package is utilizing node/polyfills/_fs/_fs_readFiles.ts as a polyfill to execute the exported function readFileSync. That function however is using Deno.readFileSync which requires --allow-read as an argument?

What is the ideal solution here? Is there another readably available option besides Deno.readFileSync to use when utilizing this polyfill? Or can we pass a flag that disables the allow-read option in this polyfill?

[ghost]

It seems that while iterating over the files within the npm package any files with the extension .js are being passed into this extension function before they passed into ops.op_require_read_file which runs the method ensure_read_permission. On the file path.

Module._extensions[".js"] = function (module, filename) {
  const content = ops.op_require_read_file(filename);

  if (StringPrototypeEndsWith(filename, ".js")) {
    const pkg = ops.op_require_read_package_scope(filename);
    if (pkg && pkg.exists && pkg.typ == "module") {
      let message = `Trying to import ESM module: ${filename}`;

      if (module.parent) {
        message += ` from ${module.parent.filename}`;
      }

      message += ` using require()`;

      throw new Error(message);
    }
  }

  module._compile(content, filename);
};

There is a method in place for .json which I am assuming is being used for package.json. However the .json within this package never calls this method.

Module._extensions[".json"] = function (module, filename) {
  const content = ops.op_require_read_file(filename);

  try {
    module.exports = JSONParse(stripBOM(content));
  } catch (err) {
    err.message = filename + ": " + err.message;
    throw err;
  }
};