Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update filoTips default #636

Merged
merged 2 commits into from
Oct 5, 2023
Merged

Update filoTips default #636

merged 2 commits into from
Oct 5, 2023

Conversation

bioimageiobot
Copy link
Collaborator

This is an automatic PR created by the @bioimageiobot regarding changes to the resource item 10.5281/zenodo.8346993.
The following version(s) will be added:

Please review the changes and make sure the new item or version(s) pass the following check list:

  • Passed the bioimage.io CI tests: static (and dynamic) validations
  • The meta information for the RDF item is complete
    • The tags are complete and describe the model
    • Naming is intuitive and descriptive, example: Multi-Organ Nucleus Segmentation (StarDist 2D)
    • Authors are provided
    • Documentation is complete
      • For models, include an overview, describe how the model is trained, what is the training data, how to use the model, how to validate the results and list the references. TODO: Model documentation template.
  • Approved by at least one of the bioimage.io admin team member.

Maintainers: @eddin022

Note: If you updated or re-uploaded another version for the current item on Zenodo, this PR won't be changed automatically. To proceed, you can do the following:

  1. Block this version, but keep looking for future versions: Edit the current resource.yaml and keep the top-level status field as accepted, but change the status under the current version to blocked.
  2. Accept this version and keep looking for future versions: Merge this PR for now.
  3. Keep proposed version(s) (and this resource in general if it is new) as pending: Close this PR without merging.

Then wait for the CI on the main branch to complete. It should detect the new version(s) and create another PR for the new version(s).

Previous PRs of this resource: none

@FynnBe
Copy link
Member

FynnBe commented Sep 19, 2023

Thank you for contributing @eddin022 !

The zenodo record is missing all files linked in the rdf.yaml.
@oeway , maybe there was an issue with bioimage.io/upload ?

For whatever reasons the files are missing, @eddin022 please add them. Let us know if you encounter any issues using bioimage.io/upload . Note that new zenodo record versions can also be created directly at zenodo.org.

One other thing I already noticed:

  • Your citation entry "Eddington C., Shwartz J., and Titus M. filoVision: using deep learning and tip markers to automate filopodia analysis." is missing a doi or url (an empty string is not valid).

@esgomezm
Copy link
Contributor

esgomezm commented Oct 4, 2023

@eddin022, I think you used the ZeroCostDL4Mic notebook for this one. When you run the export to the bioimage model zoo, the notebook generates a zip file with the model and all the content that @FynnBe addressed. Can you see that zip file? Could you please send us a screenshot of its content?
Let us know if we can help you upload the model correctly :)

@eddin022
Copy link

eddin022 commented Oct 5, 2023 via email

@FynnBe
Copy link
Member

FynnBe commented Oct 5, 2023

To fix this, should I edit the upload on Zenodo or Bioimage (or both)?

Whatever you prefer. You are updating the zenodo record either way. bioimage.io/#/upload may be more convenient to just drop the whole zip file.

I'll block this pending version 1 and look forward to see your updated version 2!

@bioimageiobot
Copy link
Collaborator Author

preview-collection-json

@FynnBe FynnBe merged commit 0a18021 into main Oct 5, 2023
@FynnBe FynnBe deleted the auto-update-10.5281/zenodo.8346993 branch October 5, 2023 09:30
@eddin022
Copy link

eddin022 commented Oct 5, 2023 via email

@FynnBe
Copy link
Member

FynnBe commented Oct 5, 2023 •
edited
Loading

No worries, very happy to see that you contribute!

Either on zenodo.org or through bioimage.io you'll create another version of the record.
Each Zenodo record has a concept DOI that is fixed and each version of that record has their own version specific DOI, see https://help.zenodo.org/faq/#versioning.

To add a new verison to a record on zenodo navigate to it while being logged in and you'll see a "new version" button below "edit" (edit is for chaning title, descripiton etc, keeping the files untouched):
image

The bioimage.io/#/upload form also allows you such an update.

here in collection-bioimage-io we now know of your filoTips model and only blocked the current, first version, not the "concept" of it. So a new version will trigger a new PR.

So, yes please upload a new version of your record.

@eddin022
Copy link

eddin022 commented Oct 16, 2023 via email

@bioimageiobot bioimageiobot mentioned this pull request Oct 17, 2023
Merged
7 tasks
@FynnBe
Copy link
Member

FynnBe commented Oct 17, 2023

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this.
Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

@oeway
Copy link
Contributor

oeway commented Oct 17, 2023

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this.
Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

I haven’t looked into the uploader yet, but with the upgrade in zenodo, I doubt it will work properly.

I suggest that hold on the task and maybe take the oppotunity to migrate the uploading system to the new one. @FynnBe This however does mean that we will need to let the upload stop for a while. What do you think?

@esgomezm
Copy link
Contributor

Hi @oeway and @FynnBe I have just checked and the uploader is not working at all. It does not upload any file anywhere. @oeway I agree with you that maybe it should be temporarily blocked until is solved.

@eddin022 I apologise for all the mess. I checked your repo and I think you directly updated the zenodo version by uploading the zip file, am I right? You did as we told you and we were not very clear so, our apologies.
Now, the content of the zip file is perfectly fine (I double-checked)! =)
We use the zip to drag and drop the content when uploading a model through the bioimage model zoo web page. If you want to update Zenodo directly as you did, you need to upload the files inside the zip independently (without being compressed). So all these files: Contours2InstanceSegmentation.ijm, sample_input_0.tif, README.md, sample_output_0.tif, cover.png, test_input.npy, keras_weights.hdf5, test_output.npy, per_sample_scale_range.ijm, tf_weights.zip, rdf.yaml, training_evaluation.csv. It will look like this:

Screenshot 2023-10-17 at 10 37 38

Please, update it and we will follow up!

@eddin022
Copy link

eddin022 commented Oct 17, 2023 via email

@FynnBe
Copy link
Member

FynnBe commented Oct 17, 2023

There is an issue with the new version you uploaded: Somehow the zenodo record holds a zip archive instead of the individual files. We currently do not support this.
Maybe @oeway already knows what's going wrong. There might be an issue with the uploader.

I haven’t looked into the uploader yet, but with the upgrade in zenodo, I doubt it will work properly.

I suggest that hold on the task and maybe take the oppotunity to migrate the uploading system to the new one. @FynnBe This however does mean that we will need to let the upload stop for a while. What do you think?

I wouldn't mind. The option to upload directly to zenodo is unaffected.

@FynnBe
Copy link
Member

FynnBe commented Oct 17, 2023

How should I proceed with uploading the other 3 models?

@eddin022 Please go directly to https://zenodo.org, login there, (go to the record you want edit) and create the (new verison of the) record as laid out by @esgomezm.

@eddin022
Copy link

eddin022 commented Oct 17, 2023 via email

@FynnBe
Copy link
Member

FynnBe commented Oct 17, 2023

I'm happy to upload them directly on zenodo, but I
would also like them to be uploaded on bioimage.io if possible. If I upload
them directly on zenodo, can we later have them available on bioimage.io as
well?

Dear Casey, apologies for our unclear communication -- we've been talking amongst ourselves so much it is hard sometimes to imagine a fresh pair of eyes on the matter.
As we currently use zenodo.org as our storage space for all models, all the bioimage.io uplaoder would do is upload to zenodo.org in your name and make sure the keyword "bioimage.io" is present.
Through this keyword our bot detects relevant zenodo records and opens a Pull Request (PR) like this one.

The upload to zenodo.org via bioimage.io has caused confusion before; therefore we decided to change this part in the near future. For now, please uplaod manually to zenodo.org and make sure to tag your recrods with the "bioimage.io" keyword. Once a PR such as this is merged they will show up on bioimage.io !

@eddin022
Copy link

eddin022 commented Oct 17, 2023 via email

@bioimageiobot bioimageiobot mentioned this pull request Oct 17, 2023
Open
7 tasks
@esgomezm
Copy link
Contributor

esgomezm commented Oct 18, 2023 •
edited
Loading

@eddin022 just to double confirm with you:
Each of your new models should be in a completely new Zenodo repository. So, each Zenodo repository (same as for papers, code or whatever), represents "A MODEL". If you change the model (as you did until now to update it or correct the format), then you need to update its corresponding Zenodo repository with a new version.
If you want to upload a new model, then you need to create a new repository by clicking on + (new upload) in Zenodo. Once you create it, make sure as @FynnBe said, that you add the bioimage.io as a community. See the shots below
Hope it's getting easy!

  • New upload to create a new zenodo repository.
Screenshot 2023-10-18 at 11 46 41 * Connect with the bioimage.io Screenshot 2023-10-18 at 11 46 56

@eddin022
Copy link

eddin022 commented Oct 18, 2023 via email

@cfusterbarcelo cfusterbarcelo mentioned this pull request Oct 25, 2023
Open
@eddin022
Copy link

eddin022 commented Nov 8, 2023 via email

@FynnBe
Copy link
Member

FynnBe commented Nov 13, 2023

Dear Casey,
I'm working on fixing the issues in #656
And I haven't seen the datasets you're mentioning. Would you be so kind to share the zenodo links or DOIs of the resources you want to contribute, then I'll check each one myself.

I'm sorry for the delay and inconvenience. We are working on a series of internal updates and recently Zenodo changes their API which threw us a bit into turmoil.

@eddin022
Copy link

eddin022 commented Nov 13, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
auto-update
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@bioimageiobot @FynnBe @esgomezm @eddin022 @oeway