Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate need for persistent caching now that bdb has been removed. #19

Closed
JohnLCaron opened this issue Jul 27, 2019 · 0 comments · Fixed by #135
Closed

Investigate need for persistent caching now that bdb has been removed. #19

JohnLCaron opened this issue Jul 27, 2019 · 0 comments · Fixed by #135

Comments

@JohnLCaron
Copy link
Contributor

It might be we still want to persist FmrcInv.xml on the server; see Unidata/netcdf-java#17.

We should survey all caching needs on the server and decide how to proceed.
@JohnLCaron JohnLCaron mentioned this issue Jul 27, 2019
Merged
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 21, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
df4fb98 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 21, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
a210042 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 23, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
29983d1 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 23, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
a586929 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 23, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
1d26a03 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
@lesserwhirls lesserwhirls mentioned this issue Jul 23, 2021
Merged
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 23, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
07a0c44 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 24, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
9a620f3 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
lesserwhirls added a commit to lesserwhirls/tds that referenced this issue Jul 26, 2021
@lesserwhirls
Implement persistent caching for FMRC GridDatasetInv
bd25df1 
Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.
greenteadigital added a commit to asascience/tds that referenced this issue Jan 20, 2022
@greenteadigital @oxelson
@lesserwhirls @dopplershift @geojs @ethanrd
Tds 5.4 snapshot (#2)
cd935fd 
* content migration and formatting

* style changes for further info callout

* finish jvm settings

* rename & move

* content migration

* added toc

* content reorg

* content dir content

* content migration

* content migration

* content reorg

* content migration

* content migration

* moved to quick start guide

* added overview section

* content migration

* Dependency upgrades

* Bump the following Bill of Materials versions:
  * netndf-java-bom (5.4.0-SNAPSHOT to 5.2.0-SNAPSHOT)
  * spring-framework-bom (5.3.1 to 5.3.4)
  * spring-security-bom (5.4.1 to 5.4.5)
  * chronicle-bom (2.20.93 to 2.21ea74)
* utilize the netcdf-java-testing-platform for testing dependency
  management
* Suppress OWAPS dependency check false positives
* Move to edal-java 1.5.0
  * this forces the minimum required version of Java from 8 to 11.

* Drop JDK 8 from GitHub Actions matrix.

* Upgrade to Gradle 6.8.3

* Migrate away from jcenter

* fixed typos

* supplemented content

* fixed broken links

* removed docs not needed in this guide

* content addition

* content reorganization

* updated security content section

* created separate section for customization

* catalog overview

* reorg of content

* clone userguide

* added tasks for new guides

* guide reconfig

* replace current -> version # for netcdf-java

* added docset_version

* replace current -> version # for tds

* changed sidebar to admin

* mv overview --> index.html

* removed netcdfjavatutorial

* add breadcrumbs include

* modify breadcrumbs layout

* breadcrumbs & side nav title css

* content change

* added all guides to docSets

* file restructure

* content reorg

* inject category titles into breadcrumbs

* breadcrumb styling

* elements for bc styling

* toc placeholder

* content migration

* rename and/or move

* added further info callout section

* table style changes

* content migration and formatting

* style changes for further info callout

* finish jvm settings

* rename & move

* content migration

* added toc

* content reorg

* content dir content

* content migration

* content migration

* content reorg

* content migration

* content migration

* moved to quick start guide

* added overview section

* content migration

* fixed typos

* supplemented content

* fixed broken links

* removed docs not needed in this guide

* content addition

* content reorganization

* updated security content section

* created separate section for customization

* catalog overview

* reorg of content

* content migration

* Remove private docs directory

Archived on google drive

* Migrate docs to new structure

* Remove buildSrc

Migrating to unidata nexus gradle plugin

* Update docset _config and sidebar files

* Add files shared between docsets

* Upgrade docs gradle build

Upgrade docs gradle build to use latest unidata jekyll plugin (and
theme), and begin using unidata nexus gradle plugin for publishing
documentation.

* Have all userguide pages use user_sidebar.yml

* Have all quickstart pages use quickstart_sidebar.yml

* Have all devguide pages use dev_sidebar.yml

* Use netCDF-Java builder interface by default.

* Migrate from NotTravis to NotPullRequest junit category

Also, add failing dap4 test to NotPullRequest - will still fail in
jenkins, but will allow us to get a baseline of passing tests on github.

* Update GitHub Actions

Split out dependency check and style check jobs into their own configs
Upgrade TDS test action to use thredds-test-action.

* Make javadoc happy with java 11.

* Use Unidata snapshot of edal-java 1.5.0 release.

* edal-java / ncWMS fixes.

* Fix thymeleaf warning about templateMode HTML5

* Chronicle-Map adjustments

* Remove chronicle-analytics transient dependency
* Implement custom seralization using BytesWriter<DatasetExt>, BytesReader<DatasetExt>, ReadResolvable<DatasetExtBytesMarshaller> instead of relying on Externalizable.

* Apply Aggregation settings to builder based Aggs

Make sure to set typical dataset mode and diskcache for
ucar.nc2.internal.ncml.Aggregation.

* Remove old documentation src.

* Migrate it subproject to tds test SourceSet

* Rework integration testing gradle infrastructure

* Don't startup H2 database instance with the TDS. Let wms handle that.

* Don't set datasetTracker object to null on close

Also, very minor cleanup in DatasetManager.

* Give epsg database time to cleanup

* Update README.txt template and create README.md for developers.

* Implement persistent caching for FMRC GridDatasetInv

Implement a InventoryCacheProvider using ChronicleMap to enable a
persistent cache for FMRC GridDatasetInv caching.

Resolves Unidata#19.

* Use cdm-test-utils over local test utils subproject.

* rename `testUtil` to `tds-test-utils`
* Use `cdm-test-utils` in place of local to see what needs to be kept in
  this repository.

* Only keep necessary classes in tds-test-utils

* Relocate package thredds to thredds.test.util

* Add ClientcatalogUtil to tds-test-utils

* Properly handle encoding when reading strings from netCDF-C

* Do not set system level file encoding

* Use correct resources path for dap4 servlet.

* Remove use of reflection to get netCDF-C library

* Set EpsgDatabasePath using URI to ensure tests work on Windows.

* Fix dap4 tests

* OPeNDAP html form changes from 4.6.x

Unidata#134, Unidata/thredds#1363

* Disable http credential caching for tests.

* Improve RadarServer tests

Improve tests to actually inspect content, not just check for 200. This
results in making the tests fail.

* Fix RadarServer initilization

Perform init logic in @eventlistener method, as afterPropertiesSet()
occurs before TdsInit.onApplicationEvent, which results in a partially
initilized service.

* Update RadarServer configs based on TdsConfig project.

* Use new netCDF-java API

No longer allow the use of the old API at runtime as a threddsConfig.xml
option or Java systems property. Also, fix a few modifier order issues.

* Clear HTTPSession credentials between tests

* Correctly handle NcML embedded inside TDS config catalogs.

* Handle netcdf element's addRecords attribute special

The new netcdf-java API does not really handle the addRecords attribute
of the netcdf element for NcML. Add a special case in DatasetManager to
deal with this.

New API IOSPs do not expose IOSP messages, so remove addRecords attribute.

* NcML addRecords attribute does not work for single dataset.

* Only use new API in tests.

* Enable use of wmsConfig.xml

During the move from ncWms 1.x to edal-java 2.x, use of the
wmsConfig.xml was not working and had to be reimplemented. Many of the
classes involved in making the custom defaults (by path, standard name,
or variable name) work was able to be reused, although porting to use
edal-java took a bit of time. A few new classes had to be created as
well.

* Update WMS documentation

Also add documentation specific to wmsConfig.xml.

* Remove CORS configuration from TDS

CORS should implemented at the servlet container or proxy server level.

* Publish tds war and tdm jar to downloads repo

* Remove the CdmrFeature service.

* Set setExpandEntities to false on SAXBuilder

See Unidata/netcdf-java#801.

* Bump spring framework and spring security deps

* Update cve suppression file to exclude those that are not applicable.

* Use netcdf-java 5.4.2 release.

* Fix integration tests that fail due to removal of CdmrFeature in Unidata#146

* Upgrade to gradle 6.9.1

* Temurin does not have java 14 binaries, so remove from github test matrix

* Fix typo

* Update and reshuffle some docs.

* Update the docs related to upgrading the TDS.

* Remove references to java 8 in favor of java 11

* Release 5.0.0-beta9

* Publish downloads under tds/version/ just like docs

* Resume work on 5.0.0-SNAPSHOT

* Fix download and doc publishing

* Publish docs under version/ and not project/version
* Increase heap space to account for memory needed for publishDownoads
task

* Remove server version info from catalog and serverInfo service.

* Upgrade to thredds-test-action@v2 and add zulu 14 to test matrix.

* Update palette selections in wmsConfig.xml to be valid ncWMS 2 names.

* Document old and new palette name mappings in release notes.

* Fix WmsDetailedConfigTest default palette check.

* update codeowners

* Fix service init sequence

Make sure the OPeNDAP and WCS init sequence happens after
threddsConfig.xml has been read. This issue was introduce after
migrating to a new version of spring and was just now noticed.

* Fetch TDS version info file from downloads.unidata.ucar.edu

* Fix topnav menu links.

* Release 5.0

* begin work on 5.1-SNAPSHOT

* update dependencies

* Fix missing OpenStreetMap attribution for NCSS maps

* fix testVersionRetrieval, which began failing after renaming latest versions to stable and maintenance

* removed link to admin guide

* Fix content-type returned by FileServer for JSON files (TDS GH Issue Unidata#161).

* Fix content-type testing (TDS GH Issue Unidata#161).

* do no wrap explicit line breaks on enum constants

* run spotless on ful project

* Minor content-type test clean-up and spotless fixes.

* Fix a bad merge and yet another spotless fix.

* Final, I hope, spotless fix.

* Spotless fix.

* Spotless fix.

* suppress new version of wire-ios CVE

* Change github action to refer to TDS, not netCDF-Java

* Change dtd location to use https.

* jdk version correction

* Turn off validation of wmsConfig.

* www --> schemas host migration

* Remove unused 4.6 wms palettes.

* spring framework upgrade

* Release 5.1

* begin work on 5.2-SNAPSHOT

* update docsets to working version, 5.2

* upgrade log4j to 2.15.0

* release 5.3

* begin work on 5.3-SNAPSHOT

* update threddsIso version

* upgrade log4j to 2.16.0

* supressing false positive owasp hit for h2-1.3.173.jar

* Upgrade to log4j 2.17.0

Use the log4j bom with enforcedPlatform in tds-platform project to allow
us to define the log4j version in one spot while controlling transitive
version resolution at the same time.

* release 5.3

* begin work on 5.4-SNAPSHOT

* bump threddsISO to snapshot

* add fixes for messed up release process (now netcdf-java 5.5.1)

* more log4j, forever log4j

* fix link to tds docs, used by thymeleaf footer

Co-authored-by: Jennifer Oxelson <[email protected]>
Co-authored-by: Sean Arms <[email protected]>
Co-authored-by: haileyajohnson <[email protected]>
Co-authored-by: Ryan May <[email protected]>
Co-authored-by: haileyajohnson <[email protected]>
Co-authored-by: geojs <[email protected]>
Co-authored-by: Ethan Davis <[email protected]>
Co-authored-by: Jennifer Oxelson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement persistent caching for FMRC GridDatasetInv lesserwhirls/tds
1 participant
@JohnLCaron