OPeNDAP URL with 'https://' fails, same OPeNDAP endpoint with 'http:' succeeds.

[noaaroland]

When trying to read an OPeNDAP URL, I get a file not found error:

x = cf.read('https://data.pmel.noaa.gov/aclim/thredds/dodsC/B10K-K20_Level2_CORECFS_integrated_collection.nc')
x

FileNotFoundError: [Errno 2] No such file or directory: 'https://data.pmel.noaa.gov/aclim/thredds/dodsC/B10K-K20_Level2_CORECFS_integrated_collection.nc'

It's not file, but I don't know of a way to indicate on the read that it is an OPeNDAP URL so I assume cf.read is supposed to figure it. I expect this to work since I have the necessary netCDF libraries installed.

There is a test for 'http:https://' in read.py, but my attempt to add a test for or 'https://' didn't work as I expected so I don't know for sure if that is the issue, however the same THREDDS endpoint is available behind our firewall with an 'http:https://' address. This url works:

y = cf.read('http:https://[inside the firewall url]/aclim/thredds/dodsC/B10K-K20_Level2_CORECFS_integrated_collection.nc')
y

[<CF Field: long_name=time-averaged Small copepod concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=S-coordinate stretching curves at RHO-points>,
 <CF Field: long_name=time-averaged Offshore euphausiid concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged On-shelf euphausiid concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged Jellyfish concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged Microzooplankton concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged Offshore large copepod concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged On-shelf large copepod concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged Ammonium concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mmol N m^-3)*m>,
 <CF Field: long_name=time-averaged Nitrate concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mmol N m^-3)*m>,
 <CF Field: long_name=time-averaged Large phytoplankton concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=time-averaged Small phytoplankton concentration, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (mg C m^-3)*m>,
 <CF Field: long_name=S-coordinate parameter, critical depth meter>,
 <CF Field: long_name=time-averaged Cop net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged EupO net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged EupS net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged Jel net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged MZL net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged NCaO net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged NCaS net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged PhL net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: long_name=time-averaged PhS net production rate, summed over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) mg C m^-2 d^-1>,
 <CF Field: ocean_s_coordinate_g1>,
 <CF Field: long_name=time-averaged potential temperature, integrated over depth(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) (Celsius)*m>,
 <CF Field: long_name=time-averaged free-surface(axis=T(2786), ncdim%eta_rho(258), ncdim%xi_rho(182)) meter>]

Platform: Linux-4.18.0-477.21.1.el8_8.x86_64-x86_64-with-glibc2.28
HDF5 library: 1.14.2
netcdf library: 4.9.2
udunits2 library: /home/users/rhs/miniforge-pypy3/envs/cf/lib/libudunits2.so.0
esmpy/ESMF: 8.4.2
Python: 3.11.6
dask: 2023.9.3
netCDF4: 1.6.4
psutil: 5.9.5
packaging: 23.2
numpy: 1.26.0
scipy: 1.11.3
matplotlib: 3.8.0
cftime: 1.6.2
cfunits: 3.3.6
cfplot: 3.2.23
cfdm: 1.10.1.2
cf: 3.15.3

[davidhassell]

Hi Roland,

Briefly (until Monday!), I thought we'd fixed this (NCAS-CMS/cfdm#268), and you're indeed using the latest cfdm library, but it seems we didn't pass this through to cf-python. I'll get it fixed after weekend and a new version is due anyway, so it'll be available very soon.

Thanks,
David