Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ngap cache #821

Merged
merged 32 commits into from
Oct 13, 2023
Merged

Ngap cache #821

merged 32 commits into from
Oct 13, 2023

Conversation

jgallagher59701
Copy link
Member

No description provided.

I used the existing BESContainer attributes to signal that access()
was going to return the cached content (the whole XML doc as a string)
and not a file name. I extended teh DMZ parser so that it can parse
strings. I added some timing to the new code along with 'hit' or 'miss'
text for the 'cache' BESDebug key. It's pretty rough still.
Copy link
Contributor

@ndp-opendap ndp-opendap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a systemic swapping of real_name in the parameters.
for example:

bool NgapContainer::get_dmrpp_cache(const string &url_key, string &real_name) 

But then it's used is like:

dmrpp_cache(get_real_name(), dmrpp_string)

Which is very confusing to me, a human. I think the semantics of these caching methods would be greatly improved if they stuck to consistent generic variable naming like:

bool NgapContainer::get_dmrpp_cache(const string &key, string &value)

Or simply got the associations correct:

bool NgapContainer::get_dmrpp_cache(const string &real_name, string &dmrpp)

@jgallagher59701 jgallagher59701 marked this pull request as ready for review October 2, 2023 02:04
The ngap.conf file explicitly sets the caches to 'true'. Also,
modified the comments in the DMR++ module to indicate that the
module uses local caches for the DDS and DAS (DAP2 metadata) but
that the DMR++ caching in currently done in the NGAP module.
…share' using in its place

For this modification, the code that managed the curl handle pool (the original
code) processed data from 20 requests for 2 variables with contiguous storage in
222ms. Removing the 'handle reuse' of the pool saw that time change to 351ms.
Using the sharing feature of curl (and the blunt force trauma lock functions)
and that time becomes 227ms. So our self-managed and the libcurl scheme are
effectively equal, with the latter having some room for better performance if
the lock functions are improved. jhrg 10/6/23
No performance testing, but the unit and regressoion tests work.
…atform

Also, moved the CurlHandlePool instance to a unique_ptr<>.
@sonarcloud
Copy link

sonarcloud bot commented Oct 12, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

warning The version of Java (11.0.15.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

@sonarcloud
Copy link

sonarcloud bot commented Oct 12, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

warning The version of Java (11.0.15.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

@sonarcloud
Copy link

sonarcloud bot commented Oct 12, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 10 Code Smells

0.0% 0.0% Coverage
0.3% 0.3% Duplication

warning The version of Java (11.0.15.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

Copy link
Contributor

@ndp-opendap ndp-opendap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dizzying array of updates. I'm glad this is working!
:shipit:

@jgallagher59701 jgallagher59701 merged commit 5641635 into master Oct 13, 2023
4 checks passed
@jgallagher59701 jgallagher59701 deleted the ngap-cache branch October 13, 2023 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants