NetCDF: Can't open HDF5 attribute

[santibravocmcc]

I am having a constant error when finally closing the netcdf file after run. It completes the execution, but for some reason it saves it corrupted. I have tried multiple times, with multiple settings, but no luck yet. Does anybody have a clue on how to resolve that? Thank you very much!

[knutfrode]

Hi, there was a similar issue on Windows with a previous version of OpenDrift.

Can you update to latest version, and see if problem persists?

[knutfrode]

Did you run the pytest suite?
You simply type pytest from commandline inside the main opendrift-folder.
This should run 244 tests, of which none should fail.

Otherwise, it would be useful if you could make a minimalistic example that reproduces this problem, i.e. using only forcing datasets within the repository, or ideally without forcing datasets at all.

[santibravocmcc]

Just ran the pytest, thanks for the suggestion. Here are the results.

===================================================================================== short test summary info =====================================================================================
FAILED tests/models/test_io.py::test_custom_result - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_buffer_length_stranding - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_export_step_interval - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_no_active_but_still_unseeded_elements - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_output_time_step - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_temporal_seed - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/test_run.py::TestRun::test_vertical_mixing - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/models/openoil/test_oillibrary.py::test_seed_metadata_output - RuntimeError: NetCDF: Can't open HDF5 attribute
FAILED tests/readers/test_xarray.py::TestXarray::test_density - RuntimeError: NetCDF: Can't open HDF5 attribute
========================================================= 9 failed, 208 passed, 26 skipped, 2 xfailed, 1650 warnings in 382.84s (0:06:22) =========================================================

Otherwise, it would be useful if you could make a minimalistic example that reproduces this problem, i.e. using only forcing datasets within the repository, or ideally without forcing datasets at all.

I am not sure if you still want me to do some tests, even though the pytest somewhat failed. Let me know if you would like for me to do a specific test or return to you a specific output that could be useful. Thanks again!

[knutfrode]

The error message from these tests indicate there is something with your netCDF/HDF libraries.
You could try the following to get a more detailed error log:
pytest -sk test_custom_result

You could also try the suggestions from this thread:
https://stackoverflow.com/questions/49317927/errno-101-netcdf-hdf-error-when-opening-netcdf-file#49317928
e.g. setting environment variable HDF5_USE_FILE_LOCKING=FALSE

[santibravocmcc]

Thank you, it worked doing this:

conda env config vars set HDF5_USE_FILE_LOCKING=FALSE
conda deactivate
conda activate opendrift
### Checking 
echo $HDF5_USE_FILE_LOCKING
>FALSE

Considering that it was not needed for Linux, maybe it is good to alert Mac users on the installation steps. Thanks a lot.

[knutfrode]

Very good.

Instead of adding complexity to installation instructions, we can let this discussion thread be a resource for others who encounter the same problem.
The problem might also not be relevant to all mac users, and may also be resolved in the future.

[santibravocmcc]

Just to alert possible viewers that the solution worked for some time, some uses actually, and then no more unfortunately.

[knutfrode]

Hi, another user is experiencing the same problem, also on Mac, but in Jupyter notebook.
Are you also having this problem in notebooks?
And did you find any solution in the meantime?

[santibravocmcc]

Hello, it has been some time since it is working again. I did a clean installation, new environment, etc. I am sharing with you the env.yaml that I created for my project which is working well, from there maybe you can have some clues.
env.yaml