Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated the nested query to be able to use the index for searching instead of a full table scan #2335

Merged
merged 2 commits into from
Nov 22, 2023

Conversation

aditya-07
Copy link
Collaborator

IMPORTANT: All PRs must be linked to an issue (except for extremely trivial and straightforward changes).
Fixes #2331

Description
Search query generated because of the has block was resulting in a full table scan instead of using an index.

The new query now is able to use this composite index of resourceType and resourceId.

Alternative(s) considered
Have you considered any alternatives? And if so, why have you chosen the approach in this PR?

Type
Choose one: Bug fix

Tested Search request

fhirEngine.search<Patient> {
  filter(TokenClientParam("active"), { value = of(true) })
  has(resourceType = ResourceType.Condition, referenceParam = Condition.SUBJECT) {
    filter(
      Condition.CODE,
      { value = of(Coding().apply { system = "http:https://snomed.info/sct"; code = "77386006" }) },
    )
    filter(
      Condition.CLINICAL_STATUS,
      { value = of(Coding().apply { system = "http:https://terminology.hl7.org/CodeSystem/condition-clinical"; code = "active" }) },
    )
  }
  sort(DateClientParam("_lastUpdated"), Order.DESCENDING)
  count = 10
  from = 0
}

The following table shows the time it took to run the above search request in the demo app for multiple iterations.

Iteration Before After
1 18.15 s 226 ms
2 14.73 s 299 ms
3 16.23 s 197 ms

Generated Query

Before change

SELECT 
  a.serializedResource 
FROM 
  ResourceEntity a 
  LEFT JOIN DateIndexEntity b ON a.resourceType = b.resourceType 
  AND a.resourceUuid = b.resourceUuid 
  AND b.index_name = '_lastUpdated' 
  LEFT JOIN DateTimeIndexEntity c ON a.resourceType = c.resourceType 
  AND a.resourceUuid = c.resourceUuid 
  AND c.index_name = '_lastUpdated' 
WHERE 
  a.resourceType = 'Patient' 
  AND a.resourceUuid IN (
    SELECT 
      resourceUuid 
    FROM 
      TokenIndexEntity 
    WHERE 
      resourceType = 'Patient' 
      AND index_name = 'active' 
      AND index_value = 'true'
  ) 
  AND a.resourceUuid IN (
    SELECT 
      resourceUuid 
    FROM 
      ResourceEntity a 
    WHERE 
      a.resourceId IN (
        SELECT 
          substr(a.index_value, 9) 
        FROM 
          ReferenceIndexEntity a 
        WHERE 
          a.resourceType = 'Condition' 
          AND a.index_name = 'subject' 
          AND a.resourceUuid IN (
            SELECT 
              resourceUuid 
            FROM 
              TokenIndexEntity 
            WHERE 
              resourceType = 'Condition' 
              AND index_name = 'code' 
              AND (
                index_value = '77386006' 
                AND IFNULL(index_system, '') = 'http:https://snomed.info/sct'
              )
          ) 
          AND a.resourceUuid IN (
            SELECT 
              resourceUuid 
            FROM 
              TokenIndexEntity 
            WHERE 
              resourceType = 'Condition' 
              AND index_name = 'clinical-status' 
              AND (
                index_value = 'active' 
                AND IFNULL(index_system, '') = 'http:https://terminology.hl7.org/CodeSystem/condition-clinical'
              )
          )
      )
  ) 
ORDER BY 
  b.index_from DESC, 
  c.index_from DESC 
LIMIT 
  10 OFFSET 0

After Change

SELECT 
  a.serializedResource 
FROM 
  ResourceEntity a 
  LEFT JOIN DateIndexEntity b ON a.resourceType = b.resourceType 
  AND a.resourceUuid = b.resourceUuid 
  AND b.index_name = '_lastUpdated' 
  LEFT JOIN DateTimeIndexEntity c ON a.resourceType = c.resourceType 
  AND a.resourceUuid = c.resourceUuid 
  AND c.index_name = '_lastUpdated' 
WHERE 
  a.resourceType = 'Patient' 
  AND a.resourceUuid IN (
    SELECT 
      resourceUuid 
    FROM 
      TokenIndexEntity 
    WHERE 
      resourceType = 'Patient' 
      AND index_name = 'active' 
      AND index_value = 'true'
  ) 
  AND a.resourceUuid IN (
    SELECT 
      resourceUuid 
    FROM 
      ResourceEntity a 
    WHERE 
      a.resourceType = 'Patient' AND a.resourceId IN (
        SELECT 
          substr(a.index_value, 9) 
        FROM 
          ReferenceIndexEntity a 
        WHERE 
          a.resourceType = 'Condition' 
          AND a.index_name = 'subject' 
          AND a.resourceUuid IN (
            SELECT 
              resourceUuid 
            FROM 
              TokenIndexEntity 
            WHERE 
              resourceType = 'Condition' 
              AND index_name = 'code' 
              AND (
                index_value = '77386006' 
                AND IFNULL(index_system, '') = 'http:https://snomed.info/sct'
              )
          ) 
          AND a.resourceUuid IN (
            SELECT 
              resourceUuid 
            FROM 
              TokenIndexEntity 
            WHERE 
              resourceType = 'Condition' 
              AND index_name = 'clinical-status' 
              AND (
                index_value = 'active' 
                AND IFNULL(index_system, '') = 'http:https://terminology.hl7.org/CodeSystem/condition-clinical'
              )
          )
      )
  ) 
ORDER BY 
  b.index_from DESC, 
  c.index_from DESC 
LIMIT 
  10 OFFSET 0

Checklist

  • I have read and acknowledged the Code of conduct.
  • I have read the Contributing page.
  • I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA.
  • I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
  • I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the style guide of this project.
  • I have run ./gradlew check and ./gradlew connectedCheck to test my changes locally.
  • I have built and run the demo app(s) to verify my change fixes the issue and/or does not break the demo app(s).

Copy link
Collaborator

@jingtang10 jingtang10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aditya-07 for this fix!

@jingtang10 jingtang10 merged commit 704d753 into google:master Nov 22, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Complete
Development

Successfully merging this pull request may close these issues.

Search query takes more time while loading service data
3 participants