feature: add cloudflare @effect/sql-d1 package

[ecyrbe]

Type

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

Add a cloudflare D1 sqlite client. D1 don't support transactions directly (only batches).
Since it was asked multiple times for it to be supported

import type { D1Database } from "@cloudflare/workers-types"
import { D1Client } from "@effect/sql-d1"
import { Config, Context, Data, Effect, Layer } from "effect"
import { Miniflare } from "miniflare"

export class MiniflareError extends Data.TaggedError("MiniflareError")<{
  cause: unknown
}> {}

export class D1Miniflare extends Context.Tag("test/D1Miniflare")<
  D1Miniflare,
  Miniflare
>() {
  static Live = Layer.scoped(
    this,
    Effect.acquireRelease(
      Effect.try({
        try: () =>
          new Miniflare({
            modules: true,
            d1Databases: {
              DB: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
            },
            script: ""
          }),
        catch: (cause) => new MiniflareError({ cause })
      }),
      (miniflare) => Effect.promise(() => miniflare.dispose())
    )
  )

  static ClientLive = Layer.unwrapEffect(
    Effect.gen(function*() {
      const miniflare = yield* D1Miniflare
      const db: D1Database = yield* Effect.tryPromise(() => miniflare.getD1Database("DB"))
      return D1Client.layer({
        db: Config.succeed(db)
      })
    })
  ).pipe(Layer.provide(this.Live))
}

Effect.gen(function*(_) {
  const sql = yield* D1Client.D1Client
  yield* sql`CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)`
  yield* sql`INSERT INTO test (name) VALUES ('hello')`
  let rows = yield* _(sql`SELECT * FROM test`)
  assert.deepStrictEqual(rows, [{ id: 1, name: "hello" }])
  yield* sql`INSERT INTO test (name) VALUES ('world')`
  rows = yield* sql`SELECT * FROM test`
  assert.deepStrictEqual(rows, [
    { id: 1, name: "hello" },
    { id: 2, name: "world" }
  ])
}).pipe(Effect.provide(D1Miniflare.ClientLive)))

Tests

Tests are using miniflare to have a local D1 instance

• Closes From Discord: @effect/sql with Cloudflare D1 #2700

[changeset-bot]

🦋 Changeset detected

Latest commit: 6444a3a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@effect/sql-d1	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[mikearnaldi]

Maybe we need a "batch" api in sql to support batch transactions?

[datner]

a batch api would just be the transaction api without injecting markers between statements, though I don't think all dbs support batching in this way. For example mssql doesn't lol

[mikearnaldi]

a batch api would just be the transaction api without injecting markers between statements, though I don't think all dbs support batching in this way. For example mssql doesn't lol

no, a batching api would allow us to specify a number of sql statements in a single shot, every sql database that supports transactions also supports such batching api.

example:

sql.batchTx(
  sql`SELECT ...`,
  sql`INSERT ...`,
  sql`DELETE...`
)

the execution can send a single batch to dbs that support it or simply run a BEGIN, ..., COMMIT.

[mikearnaldi]

a batch api would just be the transaction api without injecting markers between statements, though I don't think all dbs support batching in this way. For example mssql doesn't lol

no, a batching api would allow us to specify a number of sql statements in a single shot, every sql database that supports transactions also supports such batching api.

example:

sql.batchTx(
  sql`SELECT ...`,
  sql`INSERT ...`,
  sql`DELETE...`
)

the execution can send a single batch to dbs that support it or simply run a BEGIN, ..., COMMIT.

note that this is also common of products that put http apis on top of databases, while I believe this is a bad practice and that such dbs can't be used in most of the use cases where you'd need a transaction we should probably still support it.

@tim-smart we have to decide though what to do with the withTransaction api, how should that behave on a db that doesn't support it? the choices are:

1. make it defect
2. make it a no-op
3. make it a no-op and log a warning

[ecyrbe]

@mikearnaldi added first suggestion for now. it defects when using transactions.

[datner]

The more user-friendly choice is no-op and warn, but it might come as a surprise to the user

[ecyrbe]

My fear is that no-op + warning can go undetected to production. I prefer defect personnaly.

[mikearnaldi]

My fear is that no-op + warning can go undetected to production. I prefer defect personnaly.

Same here