Looking for guidance on how to navigate/combine Encounters and Vitals

[johnkwaters]

I am trying to fetch Encounters for a patient with their associated vitals, and not quite getting the model or pattern, looking for some guidance on where to read up on this.

First I tried this, and also Include instead of ReverseInclude, but don't seem to get any Observations inside the Encounters.

var esp = new SearchParams("subject", patientId)
    .ReverseInclude("Observations");
var encounters = await client.SearchAsync<Encounter>(esp);

Then I tried this more convoluted approach

var sp = new SearchParams("subject", patientId)
    .Include("Observation.encounter");
var observations = await client.SearchAsync<Observation>(sp);
var obsList = observations.Entry.ByResourceType<Observation>();

var vitals = (
    from o in obsList
    let code = o.Category?.FirstOrDefault()?.Coding?.FirstOrDefault()?.Code
    where code is "vital-signs" 
    select o ).ToList();

Was then going to iterate over each vital sign Obeservation and match it to the Encounter. I do get the Encounter object populated in the Observation, but it has no Identifier, so then I have a fuzzy match on Encounter.URL which does contain the Encounter ID. And this LINQ query to weed out the Observations that are vital signs seems really unwieldy!

Help me... I can tell I am not approaching this right, but don't know where to find this kind of information in the docs...

Thanks!

[mbaltus]

Why not change your search to ask only for vital signs Observations? The category code that you try to filter on is also available as search parameter:

var sp = new SearchParams("subject", patientId);
sp.Add("category", "vital-signs");

So after your two searches you will have Encounters for a Patient and you will have a list of vital signs for that Patient. Am I correct in understanding that the next steps are to determine during which Encounter a vital sign was observed/recorded, so you can set the Observation.encounter to reference the Encounter?
If so, you should be able to use the Encounter's identity to set the reference, like this:

obs.Encounter = new ResourceReference (enc.ResourceIdentity().WithoutVersion().ToString());

Not quite sure what you mean by "but it has no Identifier, so then I have a fuzzy match on Encounter.URL which does contain the Encounter ID", so maybe I've skipped over an important detail here. Please clarify further, if my approach does not work.

[johnkwaters]

But then if the patient had no vitals I wouldn't get anything right? I might go for the utility of getting everything in one call, and then narrow down the vitals locally. My other approach was doing all the calls in parallel, which was fairly fast. Any recommendations on best practice to get all the data for a patient as quickly as possible (since everything doesn't work). Use the includes, the parallel searches, or something else?

var patientTask = client.ReadAsync<Patient>($"Patient/{patientId}");
var esp = new SearchParams("subject", patientId)
    .ReverseInclude("Observations");
var encountersTask = client.SearchAsync<Encounter>(esp);
var immunizationsTask = client.SearchAsync<Immunization>(new SearchParams("patient", patientId));
var conditionsTask  = client.SearchAsync<Condition>(new SearchParams("subject", patientId));
var allergiesTask = client.SearchAsync<AllergyIntolerance>(new SearchParams("patient", patientId));
var sp = new SearchParams("subject", patientId)
    .Where($"subject={patientId}")
    .Where("category=vital-signs")
    .Include("Observation.encounter");
var observationsTask = client.SearchAsync<Observation>(sp);
var proceduresTask = client.SearchAsync<Procedure>(new SearchParams("patient", patientId));

await System.Threading.Tasks.Task.WhenAll(patientTask, encountersTask, immunizationsTask, conditionsTask, allergiesTask,
    observationsTask, proceduresTask);

[mbaltus]

But then if the patient had no vitals I wouldn't get anything right?

Correct, so you'll have to think about what you would need, and maybe do some extra client side filtering like you suggest.

I don't really have recommendations on which approach is best. Includes could be slower since this could be one big query with joins on the database, and if that's not up to it, you'll see performance drop. On the other hand, a lot of calls in parallel could lead to more network traffic, and also lots of parallel queries on the database. Maybe the FHIR community (https://chat.fhir.org) has more ideas on this. Or you could do some tests for both and time them.

You didn't ask, but I would recommend to change that esp SearchParams, since there is still an incorrect include on that. And, for each resource that can have a Patient as subject, there is a patient parameter. So all searches can be done with new SearchParams("patient", patientId). For most of your searches that means you can re-use one SearchParams if you want to make your code look a bit cleaner.

[johnkwaters]

I did a quick test, and the big search for 11 different resources took 26s, while the parallel version took 23, so they seem quite comparable.

[johnkwaters]

Yeah, end result with these changes is 25s vs 23s, program is:

`
#define xPARALLELL

using System.Diagnostics;
using System.Net.Http.Headers;
using System.Text;
using FHIR2HC;
using FHIR2HC.HC;
using Hl7.Fhir.Model;
using Hl7.Fhir.Rest;
using Microsoft.Extensions.Configuration;
using Newtonsoft.Json;
using Immunization = Hl7.Fhir.Model.Immunization;

// TODO - add rest of elements
// Call HC API to look up existing record

var builder = new ConfigurationBuilder()
.AddJsonFile($"appsettings.json", true, true);
var config = builder.Build();

var FHIRUrl = config["fhir.url"];
var username = config["fhir.username"];
var password = config["fhir.password"];

var hcUrl = config["hc.url"];
var hcUserName = config["hc.username"];
var hcPassword = config["hc.password"];
var hcMatrix = config["hc.matrix"];
var hcIncident = config["hc.incident"];
var hcAgency = config["hc.agency"];
var enablesend = bool.Parse(config["hc.enablesend"]);

var client = new FhirClient(
FHIRUrl,
new FhirClientSettings
{
Timeout = 10000,
PreferredFormat = ResourceFormat.Json,
VerifyFhirVersion = false,
PreferredReturn = Prefer.ReturnMinimal
},
new AuthorizationMessageHandler(new AuthenticationHeaderValue("Basic", Convert
.ToBase64String(Encoding.GetEncoding("ISO-8859-1")
.GetBytes($"{username}:{password}"))))
);

HttpClient hcClient = null;
if (enablesend)
{
var handler = new AuthorizationMessageHandler(new AuthenticationHeaderValue("Basic", Convert
.ToBase64String(Encoding.GetEncoding("ISO-8859-1")
.GetBytes($"{hcUserName}:{hcPassword}"))));
hcClient = new HttpClient(handler)
{
BaseAddress = new Uri(hcUrl)
};
hcClient.DefaultRequestHeaders.Add("TimeZone_StandardName", "America/Los_Angeles");
hcClient.DefaultRequestHeaders.Accept.Clear();
hcClient.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));

Console.WriteLine("Logging in to HC...");
var tokenResponse = await hcClient.GetAsync("1.0/token");
tokenResponse.EnsureSuccessStatusCode();
var token = await tokenResponse.Content.ReadAsAsync<TokenResponse>();
handler.Authorization = new AuthenticationHeaderValue("Session", token.access_token);
Console.WriteLine("Logged in to HC...");

}

var patientIds = await File.ReadAllLinesAsync("PatientIds.txt");
Console.WriteLine($"Found {patientIds.Length} records to fetch");

var saveParams = new SaveParams
{
MatrixName = hcMatrix,
EnsureChangesStayInFilter = true,
InsertedEntities = new List()
};

var counter = 0;

var sw = new Stopwatch();
sw.Start();

foreach (var patientId in patientIds)
{
var p = new InsertedEntity()
{
PatientID = patientId,
Incident_Name = hcIncident,
PT_Agency_Name = hcAgency,
ID = -(++counter)
};
saveParams.InsertedEntities.Add(p);

Console.WriteLine($"Fetching record {patientId}");

var psp = new SearchParams("patient", patientId);
var osp = psp
    .Where("category=vital-signs")
    .Include("Observation.encounter");
var observationsTask = client.SearchAsync<Observation>(osp);

#if PARALLELL
var patientTask = client.ReadAsync($"Patient/{patientId}");
var encountersTask = client.SearchAsync(psp);
var immunizationsTask = client.SearchAsync(psp);
var conditionsTask = client.SearchAsync(psp);
var allergiesTask = client.SearchAsync(psp);
var proceduresTask = client.SearchAsync(psp);
await System.Threading.Tasks.Task.WhenAll(
patientTask,
encountersTask,immunizationsTask,conditionsTask, allergiesTask,
observationsTask,
proceduresTask
);

#else
var patientsTask = client.SearchAsync(
new SearchParams("_id", patientId)
.ReverseInclude("Encounter:patient")
.ReverseInclude("Condition:patient")
.ReverseInclude("Immunization:patient")
.ReverseInclude("Procedure:patient")
.ReverseInclude("AllergyIntolerance:patient"));
await System.Threading.Tasks.Task.WhenAll(
patientsTask,
observationsTask
);

#endif

// $everything throws a server error
//var p2 = await client.FetchPatientRecordAsync(
//    ResourceIdentity.Build("Patient", patientId));

#if PARALLELL
var patient = patientTask.Result;
#else
var patients = patientsTask.Result;
var patient = patients.Entry.ByResourceType().SingleOrDefault();
#endif

var humanName = patient.Name.First();
var givenNames = string.Join(" ", humanName.Given);
Console.WriteLine($"Found {givenNames} {humanName.Family}");

p.FirstName = givenNames;
p.LastName = humanName.Family;
p.DOB = patient.BirthDateElement.ToDateTimeOffset().Value.Date;
p.Sex_Name = patient.Gender switch {
    AdministrativeGender.Female => "Female",
    AdministrativeGender.Male => "Male",
    _ => null
};

#if PARALLELL
var encounters = encountersTask.Result.Entry.ByResourceType().ToList();
#else
var encounters = patients.Entry.ByResourceType().ToList();
#endif
Console.WriteLine($" {encounters.Count} Encounters:");
foreach (var e in encounters)
{
var description = string.Join(", ", e.Type.Select( c => c.Text));
Console.WriteLine($" {description}, status {e.Status}, id: {e.Id}, from {e.Period.Start} to {e.Period.End}");
}

#if PARALLELL
var immunizations = immunizationsTask.Result.Entry.ByResourceType().ToList();
#else
var immunizations = patients.Entry.ByResourceType().ToList();
#endif

Console.WriteLine($" {immunizations.Count} Immunizations:");
foreach (var i in immunizations)
{
    Console.WriteLine($"    {i.ExpirationDate} {i.VaccineCode.Text}");
}

#if PARALLELL
var conditions = conditionsTask.Result.Entry.ByResourceType().ToList();
#else
var conditions = patients.Entry.ByResourceType().ToList();
#endif
Console.WriteLine($" {conditions.Count} Conditions):");
foreach (var c in conditions)
{
var code = c.Code.Coding.First();
Console.WriteLine(
$" {c.Code.Text} onset {c.Onset}, severity {c.Severity?.Text} ({code.Display} {code.Code})");
}

#if PARALLELL
var allergies = allergiesTask.Result.Entry.ByResourceType().ToList();
#else
var allergies = patients.Entry.ByResourceType().ToList();
#endif

Console.WriteLine($" {allergies.Count} Allergies:");
foreach (var a in allergies)
{
    var categories = string.Join(" ", a.Category);
    var code = a.Code.Coding.First();
    Console.WriteLine($"   {a.Code.Text} recorded {a.RecordedDate}, categories: {categories}, ({code.Display} {code.Code})");
    p.Allergies.Add(
        new Allergy { PTS2_ALG_Allergy_Name = a.Code.Text });
}

var observations = observationsTask.Result;
Console.WriteLine(" Observations:");

foreach (var v in observations.Entry.ByResourceType<Observation>())
{
    var codings = string.Join(", ", v.Code.Coding.Select(c => $"{c.Display} {c.Code}"));
    if (v.Value == null)
    {
        continue;
    }
    var val = v.Value switch
    {
        Quantity q => $"{q.Value} {q.Unit}",
        _ => v.Value.ToString()
    };
    Console.WriteLine($"     {codings}, effective {v.Effective} {val}, status {v.Status}, encounter {v.Encounter?.Reference}");
    if (v.Code.Text == "Body weight")
    {
        p.Weight = (int)(v.Value as Quantity).Value;
    }
}

#if PARALLELL
var procedures = proceduresTask.Result.Entry.ByResourceType().ToList();
#else
var procedures = patients.Entry.ByResourceType().ToList();
#endif

Console.WriteLine($" {procedures.Count} Procedures:");
foreach (var proc in procedures)
{
    var categories = string.Join(" ", proc.Category);
    var code = proc.Code.Coding.First();
    var period = proc.Performed as Period;
    Console.WriteLine($"   {proc.Code.Text} encounter {proc.Encounter.Display}, performed {period.Start} {period.End}, status {proc.Status}");
}

Console.WriteLine("-----------------------------------");

var msg = JsonConvert.SerializeObject(saveParams);

if (enablesend)
{
    var saveResponse = await hcClient.PostAsJsonAsync("dynamic/1.0/save", saveParams);
    saveResponse.EnsureSuccessStatusCode();

    Console.WriteLine("Logging out from HC...");
    await hcClient.GetAsync("sessions/1.0/logout");
}

}
sw.Stop();
Console.WriteLine($"Elapsed time {sw.Elapsed.Seconds}s");

Console.ReadKey();
`

[mbaltus]

Looks promising! I'll close this issue, since your questions seem to be answered, but please feel free to reopen or create a new one.