Matt Long (Dec 15 2022 at 23:01): Matt Long (Dec 15 2022 at 23:01):@Elizabeth Yankovsky installed an new environment including dask. On importing dask she gets the following error.
ModuleNotFoundError: No module named 'dask.widgets''
Has anyone else encountered this?
cc @geocat, @Elena Romashkova
Heather Craker (Dec 16 2022 at 16:00):This is might be from the Dask release on December 2nd. Did @Elizabeth Yankovsky's code work in November?
Matt Long (Dec 16 2022 at 16:00):I suspect it is an issue with the dask version
Heather Craker (Dec 16 2022 at 16:03):Let me try creating a clean environment with dask on my PC. I'll check if I get the same error
Heather Craker (Dec 16 2022 at 16:13):It works on my end with the following environment dask-test.yml. I have Dask v2022.12.0 and Python v3.11. I also tried it with Python v3.10 and that worked too
Heather Craker (Dec 16 2022 at 16:16):@Elizabeth Yankovsky Would you be able to send my your environment file so I can take a closer look? You can do this by:
conda env export > environment_name.yml where environment_name should be replaced with the name of your environment.yml that will appear in your working directory Elizabeth Yankovsky (Dec 16 2022 at 16:28):@Heather Craker thanks for your help. Here's the path to that file: /glade/u/home/eyankovsky/my-cesm-experiment/cesm-exp.yml
Heather Craker (Dec 16 2022 at 17:20):While I'm looking into how your environment is set up, a common fix is running conda update --all --prune when your chosen environment is activated. This updates all packages as much as possible given then various package dependencies. The --prune part is an optional flag that removes any unneeded packages (e.g. package dependencies that are no longer needed after uninstalling a package explicitly)
Heather Craker (Dec 16 2022 at 17:20):Can you give that a try and see if it fixes the issues?
Heather Craker (Dec 16 2022 at 17:52):It may also help to update your conda environment if you haven't already. You can do this by running conda update conda
Elizabeth Yankovsky (Dec 16 2022 at 18:29):I tried running conda update --all --prune ; this didn't help. I then tried conda update conda and got the following error (this is from within cesm-exp):
PackageNotInstalledError: Package is not installed in prefix.
prefix: /glade/u/home/eyankovsky/miniconda3/envs/cesm-exp
package name: conda
I wonder if that's the issue, that the prefix is not correct?
Heather Craker (Dec 16 2022 at 18:42):Hmm let me look into that. So far I haven't been able to recreate your environment from the .yml file
Heather Craker (Dec 16 2022 at 18:51):Would you be available for an impromptu google meet call to talk about how you made your environment? I might be able to assist more if I can see every step of your process
Heather Craker (Dec 16 2022 at 18:52): Matt Long (Dec 16 2022 at 18:54):I think @Elizabeth Yankovsky used
mamba env create -f environment.yml
@Heather Craker, why can't you recreate that environment?
Heather Craker (Dec 16 2022 at 18:55):conda is telling me that it's finding a lot of conflicts
Heather Craker (Dec 16 2022 at 18:55):I'll try mamba instead
Matt Long (Dec 16 2022 at 18:56):the environment could be pruned to just the essential packages...but it's installed successfully before
Heather Craker (Dec 16 2022 at 18:57):Mamba also gave me an error. This is very strange behavior
Elizabeth Yankovsky (Dec 16 2022 at 18:58):@Heather Craker , I'll join the google meet now and will guide you through the process
Lev Romashkov (Dec 16 2022 at 18:58):Yeah, when we were troubleshooting yesterday we tried having Elizabeth recreate an environment from a conda-lock file I generated and she was getting these errors despite our environments saying they had exactly the same package versions installed
Matt Long (Dec 16 2022 at 18:58):Probably unrelated...but we've had trouble with caching gumming things up previously. Running
mamba clean -a
should clear the cache...but I doubt that's the problem.
Matt Long (Dec 16 2022 at 18:59):Elena Romashkova said:
Yeah, when we were troubleshooting yesterday we tried having Elizabeth recreate an environment from a conda-lock file I generated and she was getting these errors despite our environments saying they had exactly the same package versions installed
That is totally wierd
Lev Romashkov (Dec 16 2022 at 18:59):Matt Long said:
Probably unrelated...but we've had trouble with caching gumming things up previously. Running
mamba clean -ashould clear the cache...but I doubt that's the problem.
Also tried this already, we basically went through the whole procedure we used to fix my issues installing this package earlier in the year
Matt Long (Dec 16 2022 at 19:00):blow away miniconda and start over
Elizabeth Yankovsky (Dec 16 2022 at 20:11):For anyone following this thread, Heather and I have gone through some troubleshooting and are still unable the issue. Here's what we tried:
Elizabeth Yankovsky (Dec 16 2022 at 20:15):(Also, when going through these steps we noticed that there is a dependency in environment.yml that's forcing us to use Python 3.7 and dask 2022.2.0. We don't know if that's a problem since it's a somewhat older version of dask)
Heather Craker (Dec 16 2022 at 20:21):If anyone wants to copy the same process, here are the exact terminal commands we used:
module load conda/latestmamba clean -amamba env create -f environment.ymlpythonimport dask.widgetsAt this point, Elizabeth gets a ModuleNotFoundError, but I do not. We have confirmed that we are using the same versions of dask (v2022.2.0) and python (v3.7)
Matt Long (Dec 16 2022 at 20:23):@Brian Vanderwende, if you have a moment, let us know what you think about this problem. It's weird.
David John Gagne (Dec 16 2022 at 20:36):Do you need to include a source activate <env_name> before calling python?
Elizabeth Yankovsky (Dec 16 2022 at 20:47):@David John Gagne yes, we've been doing conda activate cesm-exp (cesm-exp is the name of the environment that's generated by environment.yml above). Would be interesting to see if anyone else can replicate this error!
Matt Long (Dec 16 2022 at 20:47):perhaps try without using mamba?
Matt Long (Dec 16 2022 at 20:48):one difference might be that @Heather Craker appears to be using system conda and @Elizabeth Yankovsky is using miniconda?
Heather Craker (Dec 16 2022 at 20:48):David John Gagne said:
Do you need to include a source activate <env_name> before calling python?
Yes we did. I forgot to include it. Sorry about that
Matt Long (Dec 16 2022 at 20:48):I suggest pruning the environment file to just the bare bones set of packages.
Elizabeth Yankovsky (Dec 16 2022 at 20:49):@Matt Long I can try without mamba; and yes I was previously using miniconda. But today I removed my miniconda directory entirely and just use the default conda/latest
Heather Craker (Dec 16 2022 at 20:49):Matt Long said:
one difference might be that Heather Craker appears to be using system
condaand Elizabeth Yankovsky is using miniconda?
We removed commands in her .bashrc file that activated her version of miniconda. Then I had her use the module load conda/latest
Anissa Zacharias (Dec 16 2022 at 21:30):I just made the environment with conda and I didn't have any problems, so it's not a mamba thing
Elizabeth Yankovsky (Jan 11 2023 at 21:41):Just an update on this issue -- thanks to everyone for the help and suggestions!
The conflict was arising because I had several user-installed packages, specifically dask, that were creating a conflict (one can see these by doing 'pip list --user' within the activated environment). Once I removed the user-installed dask I was able to successfully import dask.widgets. :tada:
Notification Bot (Jan 18 2023 at 21:09):Deepak Cherian has marked this topic as resolved.
Last updated: May 16 2025 at 17:14 UTC