-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Will there be support for deno deploy + esm.sh #22
Comments
Hello, currently Deno / Supabase Edge Functions are not supported, but you might get far enough with NPM compat layer. Tested on Deno 1.29.1 import tiktoken from "npm:@dqbd/tiktoken";
console.log(tiktoken.get_encoding("gpt2").encode("Hello World!")); |
However, Deno Deploy will most likely not work yet, as NPM specifiers are not supported yet.
|
Digging a little bit deeper, you can use
It seems like there is a caching bug in |
I gave this a go David @dqbd, this is failing for me randomly. About 50% it initializes with success, and the other 50% it fails. My uneducated guess is that this is because they are running in limited resource serverless functions on Supabase, but I have no idea really.
|
@dqbd ... actually having specified a version in this fetch url, seems to have stabilized it... export async function getEncoder() {
await init(async (imports) => {
const req = await fetch(
"https://esm.sh/@dqbd/[email protected]/lite/tiktoken_bg.wasm" |
@arthur5005 sounds like you've solved this. FWIW, we also use the Worth noting JSON imports should be fixed now as of last week, so |
Can the Deno entry in Compatibility section in https://github.com/dqbd/tiktoken?tab=readme-ov-file#compatibility please be fixed? This linked issue is only regarding Deno Deploy, normal WASM tiktoken works perfectly fine in Deno itself (when using it as a runtime, similar to Node or Bun). |
I tried to load the npm package with esm.sh but it gives an error:
here is the import statement
I want to use it in supabase edge functions they use deno deploy and it does not support npm packages
Another error:
The text was updated successfully, but these errors were encountered: