Fix get_device_memory_ids #1305
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
StringColumns used to store both the character data and offsets column in their "children" slot which is not inspected by get_device_memory_ids. A recent cudf change moved the character data to the `.data` property, so it now shows up. Adapt expected number of buffers in the test accordingly.
cudf Columns should be treated specially, since they hide device buffers in mask, data, and children properties.
pentschev
approved these changes
Jan 18, 2024
Comment on lines
+314
to
+321
| # Buffers are: | ||
| # 1. int data for objects[0].a | ||
| # 2. mask data for objects[0].a | ||
| # 3. int data for objects[0].b | ||
| # 4. int data for objects[0].index | ||
| # 5. int data for objects[1].levels[0] | ||
| # 6. char data for objects[1].levels[1] | ||
| # 7. offset data for objects[1].levels[1] |
There was a problem hiding this comment.
Should we assert their types instead of listing them as a comment?
There was a problem hiding this comment.
get_device_memory_ids just gathers DeviceBuffers which are untyped. This comment is so that a future reader of the test knows where the magic number 7 comes from.
madsbk
approved these changes
Jan 18, 2024
|
Thanks @wence- ! |
|
/merge |
younseojava
pushed a commit
to ROCm/dask-cuda-rocm
that referenced
this pull request
Apr 16, 2024
A recent change to the way `StringColumn`s are implemented in cudf threw up that we were never correctly determining the number of device buffers belonging to cudf columns if they had children (e.g. list and struct columns) or masks (any nullable column). Handle those cases and update the test. Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Peter Andreas Entschev (https://github.com/pentschev) - Mads R. B. Kristensen (https://github.com/madsbk) URL: rapidsai#1305
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A recent change to the way
StringColumns are implemented in cudf threw up that we were never correctly determining the number of device buffers belonging to cudf columns if they had children (e.g. list and struct columns) or masks (any nullable column). Handle those cases and update the test.