Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix relation metadata indexing in nidx + old index #2770

Merged
merged 4 commits into from
Jan 9, 2025

Conversation

carlesonielfa
Copy link
Contributor

@carlesonielfa carlesonielfa commented Jan 9, 2025

Description

Fixes small bug in nidx and old index that prevented relation metadata from being stored properly

How was this PR tested?

  • Added unit tests
  • Extended an integration test to check for metadata

Copy link

codecov bot commented Jan 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.78%. Comparing base (09763ac) to head (19faab6).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2770      +/-   ##
==========================================
- Coverage   86.83%   86.78%   -0.05%     
==========================================
  Files         384      384              
  Lines       24308    24308              
==========================================
- Hits        21107    21096      -11     
- Misses       3201     3212      +11     
Flag Coverage Δ
nucliadb 74.60% <ø> (-0.07%) ⬇️
nucliadb-ingest 40.65% <ø> (ø)
nucliadb-reader 42.66% <ø> (-0.02%) ⬇️
nucliadb-search 43.48% <ø> (ø)
nucliadb-standalone 47.36% <ø> (ø)
nucliadb-train 45.62% <ø> (ø)
nucliadb-writer 46.10% <ø> (ø)
nucliadb_dataset 55.45% <ø> (ø)
nucliadb_models 83.29% <ø> (ø)
nucliadb_sdk 80.11% <ø> (ø)
nucliadb_sidecar 89.03% <ø> (ø)
nucliadb_telemetry 86.56% <ø> (ø)
nucliadb_utils 83.77% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@carlesonielfa carlesonielfa changed the title Fix NIDX relation metadata indexing Fix relation metadata indexing in nidx + old index Jan 9, 2025
@@ -65,7 +65,7 @@ pub fn index_relations(

if let Some(metadata) = relation.metadata.as_ref() {
let encoded_metadata = metadata.encode_to_vec();
new_doc.add_bytes(schema.label, encoded_metadata);
new_doc.add_bytes(schema.metadata, encoded_metadata);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦

@@ -37,7 +37,7 @@ pub fn index_relations(
let source = relation.source.as_ref().expect("Missing source");
let source_value = source.value.as_str();
let source_type = io_maps::node_type_to_u64(source.ntype());
let soruce_subtype = source.subtype.as_str();
let source_subtype = source.subtype.as_str();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

assert_eq!(metadata.source_end, Some(10));
assert_eq!(metadata.to_start, Some(11));
assert_eq!(metadata.to_end, Some(20));
assert_eq!(metadata.data_augmentation_task_id, Some("mytask".to_string()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to get a test as a bonus! 🙏

@carlesonielfa carlesonielfa merged commit 4f491f5 into main Jan 9, 2025
64 checks passed
@carlesonielfa carlesonielfa deleted the fix-nidx-relation-metadata-index branch January 9, 2025 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants