Clean up imports for dask>2024.12.1 support
#1424
Conversation
rjzamora
added
bug
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
/ok to test |
rjzamora
added
3 - Ready for Review
dask_cuda/__init__.py
Outdated
| "or None, or downgrade RAPIDS to <=24.12." | ||
| ) | ||
| try: | ||
| if not dd._dask_expr_enabled(): |
There was a problem hiding this comment.
I'm not sure when/if the _dask_expr_enabled attribute will be removed from the dask.dataframe module. However, when it is removed, we don't need to worry about query-planning being disabled, because that version of dask won't include the legacy API anyway.
| from dask.dataframe.shuffle import group_split_dispatch, hash_object_dispatch | ||
| from dask.dataframe.dispatch import group_split_dispatch, hash_object_dispatch |
There was a problem hiding this comment.
All dispatch functions have been centralized in the dispatch module for a long time now. Many changes in this PR are just using that preferred module.
| if not DASK_EXPR_ENABLED: | ||
| raise ValueError( | ||
| "Dask-CUDA no longer supports the legacy Dask DataFrame API. " | ||
| "Please set the 'dataframe.query-planning' config to `True` " | ||
| "or None, or downgrade RAPIDS to <=24.12." | ||
| ) |
There was a problem hiding this comment.
Moved this logic into explicit_comms.dataframe.shuffle since that's really the only place in Dask-CUDA where dask-expr matters.
rjzamora
added
5 - Ready to Merge
|
/merge |
Follow up to #1417
Cleans up some imports (some of which don't work for
dask>2024.12.1).