bug: Clinical Rules - dates and filters

[hanksterr7]

My topic # 5
Adding an issue for discussing the role of dates in evaluating demographic filters

Initially, my contention was that when evaluating rule filters on an individual patient (for populating Clinical Reminders widget in the patient's dashboard, or the Clinical Reminders details page), dates (other than "today") and intervals should not be part of that calculation

In looking at this more closely, the problem really is that demographic filters are currently being evaluated against 3 timepoints, based on a rule's warning and past due interval values. I contend that doing these three calculations, each with different timepoints, is not appropriate. Filters should be evaluated once for each rule, at the timepoint passed to test_filter() (and test_rules_clinic()) as $dateTarget.

I looked at the rule filter definitions in CDR_Rules_tables. None appear to require performing a date-based calculation. I do agree that if one is running a report, and wanting the analysis to be based on some point in time in past, then some filter calculations might need a time reference. Example would be: what was the reminder status on a patient if the report were run 3 years ago. For that, the age filters would need to calculate the patient's age as of 3 years ago. Similarly, one might want to know what diagnoses were established in the patient as of 3 years ago.
I don't see anything in the filter syntax in the rule_filter table that defines a timepoint, or an interval. For example, I could imagine a filter that wants to find patients who had diabetes added to the diagnosis list sometime in the past 12 months (and not just at any time in the past). There is nothing in the editor gui that allows specifying this interval, and I don't see a way to store this lookback period length in the filter definition in the rule_filter table.

test_rules_clinic() takes $dateTarget as a parameter. If valued, and if $mode is not 'report', then calculate_reminder_dates() takes $dateTarget and calculates $target_dates, which are based on the warning and past due intervals specified for the rule.
For example, if $dateTarget is "today", and warning: 1 month, past due: 2 months, then $target_dates will have values: T+1M, T, and T-2M. These 3 values are used to loop through the calls to test_filter() and test_targets(), and are called $dateFocus when passed to these functions.

I have no issue with using these three $dateFocus values when calling test_targets(). But I don't see a value in passing each of the three $dateFocus values to test_filter(). I do see value to passing just $dateTarget to test_filter(), and evaluating the filter as of that date.

test_filter() evaluates a filter against the passed $dateFocus value (and calls this value $dateTarget in test_filter() ). It uses its $dateTarget value in age calculations. It also passes its $dateTarget value to database_check(), lists_check() and procedure_check().

In these check functions, it examines an item relative to this $dateTarget value

So, in checking to see if an ICD code exists (via lists_check()), it compares the lists row's date, begdate and enddate values against $dateTarget. It is thus checking to see if the ICD code was defined and valid at the time of $dateTarget, via this logic, with the three ?s assigned to the current value of $dateTarget passed to test_filter():

"AND ( (begdateIS NULL ANDdate<=?) OR (begdateIS NOT NULL ANDbegdate<=?) ) " . "AND ( (enddate IS NULL) OR (enddateIS NOT NULL ANDenddate>=?) )

The problem with this approach is:
-- the three $target_dates values were defined, based on the warning and past due interval values, in order to determine if the rule was "not due", "due soon", "due" or "past due"
-- suppose an ICD code was entered 2 weeks ago and did not have a begdate or enddate associated with it, just a "date" value of 2 weeks ago
-- on the first call to test_filter(), $dateTarget will be T+1M. The date-based test will succeed and the filter will be "true"
-- on the second call to test_filter(), $dateTarget will be T and the filter will be "true"
-- on the third call to test_filter(), $dateTarget will be T-2M. date will not be less than T-2M, and the filter will fail, so the rule will not be considered applicable during the attempt to see if the rule status is "past due"

This problem can be solved by passing $dateTarget, instead of $dateFocus, to test_filter() when looping in test_rules_clinic(). It is fine to pass $dateFocus to test_target(), but not test_filter(). This is not what is in my code in the PR, so I will have to update the PR code

Note that this approach will execute each a call to test_filter() 3 times, each time with the same parameters, so the system is doing 3x the work required in evaluating filters

[hanksterr7]

I also noted this in the definition of rule_filter in the CDR Rules Tables page
``required_flag tinyint(1) NOT NULL DEFAULT '0' COMMENT '0 is required and 1 is optional',

Note that the description of this field says that 0 means required, and 1 means optional. The logic treats this field the opposite way, with 1 being requried, and 0 being optional

Your insertions into this table make this apparent. The ICD codes have a required_flag of zero (meaning optional, or any of them needs to be true), whereas the age_min, encounter_mina nd measure_period all have a required_flag value of 1 (meaning all are required to be true)
INSERT INTO rule_filter VALUES ('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','CUSTOM::HTN'),('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','ICD9::401'),('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','ICD9::402'),('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','ICD9::403'),('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','ICD9::404'),('rule_htn_bp_measure_cqm',1,0,'filt_lists','medical_problem','ICD9::405'),('rule_htn_bp_measure_cqm',1,1,'filt_age_min','year','18'),('rule_htn_bp_measure_cqm',1,1,'filt_encounter_min','','2'),('rule_htn_bp_measure_cqm',1,1,'filt_measure_period','year','1'),

I can fix this description of this field as part of the PR. Is related to my topic # 2

[bradymiller]

hi @hanksterr7 , Agree with analysis and plan for both these posts (looks like you separated the second post into a separate issue which makes sense). For post 1, hopefully we can work things so that only need to run test_filter() 1 time.

[hanksterr7]

Ok, I have updated the code in clinical_rules.php to execute the call to test_filter() only once (instead of 3 times), and to pass $dateTarget to test_filter() instead of $dateFocus. Seems to be working well

[bradymiller]

hi @hanksterr7 ,
Recommend separating post 1 code into its own PR. Goal is to be as surgically precise as possible on the code changes since the underlying code is complicated (ie. don't include log debug stuff in the code that will go into the codebase (ok to have them while coding, just need to remove them for the final product) and avoid changes to formatting of non-related code like the unrelated sql query changes). Definitely would include your explanatory comments in the code since they are very helpful. Regarding debugging, there is an awesome tool called Xdebug which is built into the easy dev docker environment if you ever go that route (note the debug log method you are using is also ok and is something I used for years until I became hooked by xdebug :) ).