Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better DB interface DX #1832

Open
2 tasks
Harjot1Singh opened this issue Nov 7, 2021 · 0 comments
Open
2 tasks

Better DB interface DX #1832

Harjot1Singh opened this issue Nov 7, 2021 · 0 comments
Labels
ϟ Type Epic Describing a big goal. A collection of issues. Should never end up in an iteration.

Comments

@Harjot1Singh
Copy link
Member

Harjot1Singh commented Nov 7, 2021

Motivation

As the database has grown and our usage has become more refined and clear, it would be easier to access the database in json format. In addition since the content is mostly static, it would help to have the higher performance of a document based db. Lastly, since setting out to work on database, it was not clear that we would eventually have a mobile app. Our current options do not work well for that.

Goal

  • Create a new database that is better for APIs, mobile apps (including delta sync), and static content (including growing/partial db "collections" document stores).
  • Easier to use in JSON, not requiring building sqlite3 binaries or any kind of binary at all in javascript/python any modern language that can leverage JSON.
  • Set up content to be based off of existing publications/historic sources. Needs to account for variable line count (2 lines in source A, 1 line in source B), re-ordering of lines (e.g. manglacharan before or after in various sources), and applying vishraams to varying pad-ched interpretations.
  • Lastly, as a bonus point to discuss: to include non-conforming content such as long form translations of multiple lines. (Today we split up these translations as we see fit, but for example some of the original sources will do translations after an entire pauri). This would in the future be potentially better than serving pdf images of the content to save bandwidth.
  • Split up codebases/repos so that database is just content, tooling to go elsewhere?

Example

Line

{
  "id": "0NVY",
  "content": [
    {
      "name": "SGPC_2018",
      "value": "<> siq nwmu krqw purKu inrBau inrvYru; Akwl mUriq AjUnI sYBM gur pRswid ]"
    },
    {
      "taksal":  "<> siq nwmu krqw purKu inrBau inrvYru; Akwl mUriq AjUnI sYBM gur pRswid ]"
    }
  ]
}

Stories/Tasks

@Harjot1Singh Harjot1Singh added the ϟ Type Epic Describing a big goal. A collection of issues. Should never end up in an iteration. label Nov 7, 2021
@Harjot1Singh Harjot1Singh changed the title New DB Better DB interface DX Jul 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ϟ Type Epic Describing a big goal. A collection of issues. Should never end up in an iteration.
Projects
Status: In Refinement
Development

No branches or pull requests

1 participant