Stream: python-dev
Topic: pooch error in latest pop_tools
Keith Lindsay (Apr 20 2020 at 18:35):
I updated to the latest pop_tools and get the following error when I run import pop_utils:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-1-4db41a978446> in <module>
10 import esmlab
11 import esmlab_regrid
---> 12 import pop_tools
13
14 from gen_remap_weights_1d import gen_remap_weights_1d
/gpfs/fs1/work/klindsay/analysis/pop-tools/pop_tools/__init__.py in <module>
4 from pkg_resources import DistributionNotFound, get_distribution
5
----> 6 from .datasets import DATASETS
7 from .eos import compute_pressure, eos
8 from .fill import lateral_fill, lateral_fill_np_array
/gpfs/fs1/work/klindsay/analysis/pop-tools/pop_tools/datasets.py in <module>
14 base_url='ftp://ftp.cgd.ucar.edu/archive/aletheia-data/cesm-data/ocn/',
15 )
---> 16 DATASETS.load_registry(pkg_resources.resource_stream('pop_tools', 'data_registry.txt'))
17
18
/glade/work/klindsay/miniconda3/envs/analysis_dev/lib/python3.7/site-packages/pooch/core.py in load_registry(self, fname)
342
343 """
--> 344 with open(fname) as fin:
345 for linenum, line in enumerate(fin):
346 elements = line.strip().split()
TypeError: expected str, bytes or os.PathLike object, not _io.BufferedReader
I don't know what to make of this. Any suggestions on how to avoid this error, or how to further diagnosis the problem?
Anderson Banihirwe (Apr 20 2020 at 18:43):
@Keith Lindsay,
What version of pooch are you running?
I just tried importing pop_tools locally and everything seems to be working
In [1]: import pop_tools In [3]: exit
pop-tools on master via 🐍 v3.7.6 via C analysis via 89%/55% took 38s ❯ conda list pooch (analysis) 12:41:51 # packages in environment at /Users/abanihi/opt/miniconda3/envs/analysis: # # Name Version Build Channel pooch 1.1.0 py_0 conda-forge
Keith Lindsay (Apr 20 2020 at 18:52):
conda list pooch returns
# packages in environment at /glade/work/klindsay/miniconda3/envs/analysis_dev: # # Name Version Build Channel pooch 0.2.1 py37_1000 conda-forge
I'll try updating it. Fingers crossed that other stuff doesn't break when I update pooch.
I tried to clone my conda environment with the idea of updating pooch in the clone. But the clone failed, so I don't see a safe/reversible way to update pooch.
Anderson Banihirwe (Apr 20 2020 at 20:15):
Do you have another environment that has pop-tools installed in it?
Anderson Banihirwe (Apr 20 2020 at 20:15):
Can you try the following and see if it works:
In [1]: import pkg_resources In [2]: pkg_resources.resource_stream('pop_tools', 'data_registry.txt') Cannot write to data cache folder '/glade/p/cesmdata/cseg'. Will not be able to download remote data files. Use environment variable 'CESMDATAROOT' to specify another directory. Out[2]: <_io.BufferedReader name='/glade/work/abanihi/softwares/miniconda3/envs/analysis/lib/python3.7/site-packages/pop_tools/data_registry.txt'>
Keith Lindsay (Apr 20 2020 at 20:43):
My other environments that have pop-tools are old enough that they don't have the file data_registry.txt.
I'm about to give up on conda updating pooch only. It's been trying for nearly 2 hours to resolve conflicts.
I think I will try to either back up to the older tag of pop-tools that I previously used,
or start a new environment for the notebook that I'm trying to write.
Anderson Banihirwe (Apr 20 2020 at 20:53):
I tried cloning your environment, but I ran into some issues with packages installed from PyPI:
abanihi at casper26 in ~ $ conda create --name analysis_dev_clone --clone /glade/work/klindsay/miniconda3/envs/analysis_dev Source: /glade/work/klindsay/miniconda3/envs/analysis_dev Destination: /glade/work/abanihi/softwares/miniconda3/envs/analysis_dev_clone PackagesNotFoundError: The following packages are missing from the target environment: - zipp==0.4.0=pypi_0 - seawater==3.3.4=pypi_0 - cfgrib==0.9.6.2=pypi_0 - importlib-metadata==0.9=pypi_0 - typing==3.6.6=pypi_0 - pop-tools==0.0.post19=dev_0 - toml==0.10.0=pypi_0 - esmlab-regrid==2019.5.2.dev8+g815fd28=dev_0 - gitpython==2.1.11=pypi_0 - intake-esm==2019.4.26.1.post15=pypi_0 - watermark==1.8.1=pypi_0 - esmlab==2019.4.27=pypi_0 - virtualenv==16.5.0=pypi_0 - nodeenv==1.3.3=pypi_0 - docrep==0.2.5=pypi_0 - cfgv==1.6.0=pypi_0 - gitdb2==2.0.5=pypi_0 - ncar-jobqueue==2019.3.8=pypi_0 - nbdime==1.0.5=pypi_0 - colorama==0.4.1=pypi_0 - dask-jobqueue==0.4.1+30.g779bea3=pypi_0 - smmap2==2.0.5=pypi_0 - pre-commit==1.15.2=pypi_0 - identify==1.4.2=pypi_0 - msgpack-python==0.5.6=pypi_0 - xcollection==0+untagged.60.g1905fea=pypi_0 - aspy-yaml==1.2.0=pypi_0
Keith Lindsay (Apr 20 2020 at 20:59):
I got the same errors when I attempted to clone the environment myself.
It's unclear to me where the packages are missing from.
Anderson Banihirwe (Apr 20 2020 at 21:03):
Try the following
Anderson Banihirwe (Apr 20 2020 at 21:03):
conda list --explicit > spec-file.txt
Anderson Banihirwe (Apr 20 2020 at 21:03):
conda install --name myenv --file spec-file.txt
Deepak Cherian (Apr 20 2020 at 21:30):
anecdotally, if your environment is too old, updating single packages makes everything fall flat. There are too many possible solutions (I think). Usually conda update --all will complete in a reasonable amount of time.
Last updated: Jan 30 2022 at 12:01 UTC