ControlNET SDXL - bucketing ?

[alelordelo]

Does the ControlNET SDXL training support bucketing for images with multi resolution and aspect ratio?

[okotaku]

@alelordelo Yes!
You can change this line https://github.com/okotaku/diffengine/blob/main/configs/_base_/datasets/fill50k_controlnet_xl.py#L1-L28 .

train_pipeline = [
    dict(type="SaveImageShape"),
    dict(
        type="MultiAspectRatioResizeCenterCrop",
        sizes=[
             [640, 1536], [768, 1344], [832, 1216], [896, 1152],
             [1024, 1024], [1152, 896], [1216, 832], [1344, 768], [1536, 640]
             ],
        interpolation='bilinear',
        keys=["img", "condition_img"]),
    dict(type="RandomHorizontalFlip", p=0.5, keys=["img", "condition_img"]),
    dict(type="ComputeTimeIds"),
    dict(type="torchvision/ToTensor", keys=["img", "condition_img"]),
    dict(type="DumpImage", max_imgs=10, dump_dir="work_dirs/dump"),
    dict(type="torchvision/Normalize", mean=[0.5], std=[0.5]),
    dict(
        type="PackInputs",
        input_keys=["img", "condition_img", "text", "time_ids"]),
]
train_dataloader = dict(
    batch_size=2,
    num_workers=4,
    dataset=dict(
        type="HFControlNetDataset",
        dataset="fusing/fill50k",
        condition_column="conditioning_image",
        caption_column="text",
        pipeline=train_pipeline),
    sampler=dict(type="DefaultSampler", shuffle=True),
    batch_sampler=dict(type='AspectRatioBatchSampler')
)

[alelordelo]

thanks @okotaku ! I didn't find any docs on how to train COntrolNET with custom dataset... The one I found is from HF with the Circle dataset. If you can provide any info on how to train I highly appreciate!

[okotaku]

@alelordelo
I have added documentation on how to prepare the dataset for ControlNet.

#100

Comments are welcomed ;)

[okotaku]

I have also added on how to use Aspect Ratio Bucketing.

#101