Client dissambiguation for HTTP and DODS gets wrong when HTTP response is 403

[cofinoa]

Client dissambiguation for HTTP and DODS when a dataset request an authentication challenge raises an exception if credential are wrong, i.e. server HTTP response code is 401. But if credentials are right but user is not authorized, i.e. server response code is 403 (forbidden), the dissambiguation doesn't rise the exception like in 401.

The dissambigution assumes non-DODS server and tries to open the dataset as regular HTTP resource, raising a 400 HTTP response code from DODS server.

[cofinoa]

This relates with pull request #547

[ethanrd]

@cofinoa What client software are you using? I'm guessing netCDF-Java?

When the netCDF-Java library is handed a dataset URL to open, it tests whether it is a DAP URL (here [1]) by appending a ".dds" to the URL and making a HEAD request. If it gets a 200 (OK) response and there is a "Content-Description" header containing "dods-dds" or "dods-das", it decides it is a DAP/DODS dataset URL. If it gets a 401 (Unauthorized) response, it throws an IOException. All other responses, including 404 (Not Found) and 403 (Forbidden), proceed on to try other protocols.

@DennisHeimbigner Is user authentication left to the client that is calling the NetcdfDataset.open(location) method? I'm not seeing any indication of that in the method documentation. But I'm also not seeing where it is handled.

[1] https://github.com/Unidata/thredds/blob/5.0.0/cdm/src/main/java/ucar/nc2/dataset/DatasetUrl.java#L386

[cofinoa]

@ethanrd I'm using netcdf-java 4.6.5.

Yes, but I have made it's also to consider the 403 as IOException like 401.

The problem is that if a real DAP URL response is 403 then the netcdf-java client it's trying the URL without the .dds and then the DAP server is returning a 400 (bad request) response code.

My concerns it's also with the message returned with the IOExceptions. In most of the response codes (i.e. 404 or 400) the message contains the returned code by the server, but for the 401 this is not returned. I have made the pull-request modifying the IOException messages in a more coherent form.

@DennisHeimbigner may have a better response, but what I'm using for authentication is the CredentialsProvider which are managed internally by the HTTPSession, like is been use by the ToolsUI app.

[1] https://github.com/Unidata/thredds/blob/5.0.0/ui/src/main/java/ucar/nc2/ui/ToolsUI.java#L5828