Only last 50 reports for contact are provided to contact-summary calculation

[jkuester]

Describe the bug
The contact-summary calculation logic is provided with an array of the reports associated with the contact (and any of the contact's person children). Unfortunately, because of the way the code was implemented (at least as far back as 3.9) the number of reports that would be retrieved for the contact and passed to the contact-summary calculation was capped at 50. So, the contact-summary for contacts associated with >50 reports, would not have the full context of all that contacts reports.

To Reproduce

1. Register a pregnancy for a contact
2. See that a pregnancy card is shown for the patient based on their contact-summary calculation
3. Submit 50 more reports for the contact
4. See that the pregnancy card is no longer shown

Expected behavior
The search.service gets used in a number of situations where it probably makes sense to have the 50 report limit (e.g. when displaying reports for the contact on the contact's profile page, we probably do not need to show an unlimited number of reports). However, for the contact-summary calculation, for the sake of accuracy, we should be providing all the reports associated with that contact.

Also, the contact-summary docs should be updated to note which version lifted the 50 report limit.

Additional context
https://github.com/medic/config-muso/issues/1050
Forum thread

Fixed 6-April-2024

The fix for this issue was merged.

• Now the calculation of the contact-summary fetches and uses 500 reports.
• However, only 50 reports are displayed in the contact detail page (For performance reasons, we don’t want to app freezing as it fetches the max number of reports (500) all at once).

[Benmuiruri]

Reproducing the error

Hi @ralfudx I have started working on this issue. First thing I have done is reproduce the error. (I have a test plan at the bottom where I welcome your input / comment)

1. I used the default sample design in the test data generator to create a household with a woman and submitted two reports.

Video showing created woman has pregnancy card when reports are less than 50

Screen.Recording.2024-03-27.at.21.04.44.mov

2. I used the add-to-existing-heirarchy sample design to add 50 more reports to the contact from 1 above.

Video showing pregnancy card for woman does not appear when reports than 50 reports and the pregnancy report is not part of the 50

Screen.Recording.2024-03-27.at.21.13.38.mov

@jkuester please confirm my understanding and reproducing of the error is correct, when you get a chance. Thank you 😄

Test Plan

• After working on the fix on the contact-summary calculation my test plan is the following:

1. Manually I will test it like I did above using the test generators and making sure that even after 50 reports, you can still see the Pregnancy Card.
2. For the automated testing, I am thinking of an e2e test that submits > 50 reports then inspects the page to expect the pregnancy card element/ class to be present.
3. I am still thinking about the solution implementation so I'm yet to know whether I can write a unit test for it.
4. Do you have any suggestions to add on manual / automated testing ?

Technical solution (Work in Progress)

[jkuester]

👍 your recreation steps are exactly correct!

[latin-panda]

@Benmuiruri and @ralfudx you can also consider adding contact-summary fields that calculate based on reports and display them in the UI: Numb. Associated pregnancies reports: 150, Numb. Associated delivery reports: 44. This would easily show that the contact summary is getting all the associated reports.

This work should not affect tasks, targets, or any element displayed in this view.

This work will probably impact performance. We need to measure the before and after:

• Contact-summary calculation: each field will go through all the reports, and for example, MoH Kenya has many contact-summary fields (# fields x # performance time)
• Page render time

We have telemetry measuring this, please post a comparison table: when having 50 reports [before, after], when having #### reports [before, after].

@michaelkohn @n-orlowski since we are working to retrieve +50 reports. Is that okay for the contact details view > reports section > view all tab? I wonder if we keep this as showing only 50 items, or if we add a search/filter to display more (outside of this ticket scope).

[Benmuiruri]

The fix for this issue was merged.

• Now the calculation of the contact-summary currently uses all fetched 500 reports.
• However, only 50 reports are displayed in the contact detail page (For performance reasons, we don’t want to app freezing as it fetches the max number of reports (500) all at once).
• I have opened an issue to implement pagination in order to efficiently load All reports associated with a contact.