fix: inconsistencies between API schema and class validators

[emigun]

Description

There are inconsistencies between the OpenApi schema and the class validators that this PR aims to fix.

Motivation

For the SDKs to work properly the api schemas need to be correct

Fixes

• setting required to the correct value in @ApiProperty
• adding class validators where some were missing (some may still be left, see discussion below)

More to discuss

• In update-dataset-obsolete.dto.ts, there are no defaults set at (ca) lines 224 (techniques), 237 (sharedWith), 251 (relationships), 262 (datasetlifecycle).
• The last five fields in update-raw-dataset-obolete.dto.ts does not have the ApiProperty decorator. Just forgotten?
• new: Should we just let the openapi plugin generate schema from decorators and type info? (see comment below)

Tests included

• Included for each change/fix?
• Passing?

Documentation

• swagger documentation updated (required for API changes)
• official documentation updated

[emigun]

Perhaps we should consider using https://docs.nestjs.com/openapi/cli-plugin instead of annotating manually?

It seems already enabled?

[emigun]

Perhaps we should consider using https://docs.nestjs.com/openapi/cli-plugin instead of annotating manually?

It seems already enabled?

Yes, it enabled. I have tested it a bit and it seems to work nicely.

A fairly advanced example:
Let's change

  @ApiProperty({
    type: "array",
    items: { $ref: getSchemaPath(TechniqueClass) },
    required: false,
    default: [],
    description: "Stores the metadata information for techniques.",
  })
  @IsOptional()
  @IsArray()
  @ValidateNested({ each: true })
  @Type(() => CreateTechniqueDto)
  readonly techniques?: TechniqueClass[] = [];

to (remove ApiProperty)

  @IsOptional()
  @IsArray()
  @ValidateNested({ each: true })
  @Type(() => CreateTechniqueDto)
  readonly techniques?: TechniqueClass[] = [];

That would change the openapi json from:

"techniques": {
  "type": "array",
  "items": {
    "$ref": "#/components/schemas/TechniqueClass"
  },
  "default": [],
  "description": "Stores the metadata information for techniques."
},

to

"techniques": {
  "default": [],
  "type": "array",
  "items": {
    "$ref": "#/components/schemas/TechniqueClass"
  }
}

As you can see everything was generated (except description, see below)

I have tried with arrays, dates and ordinary string/number and it seems to work. It also reads the ? for optional fields and correctly identifies the required ones.

We could change nest-cli.json to:

  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "plugins": [{
      "name": "@nestjs/swagger",
      "options": {
        "introspectComments": true
      }
    }]
  }
}

in order to generate the field "description" based on a comment above. Or you could still use

@ApiProperty({
    description: "This is a test description"
  })

if you prefer that. The other fields like type, items, default and others will still be generated with the plugin.

tldr; We could probably get rid of all the ApiProperty decorators and the openapi spec will automatically be generated correctly for the fields. If some doesn't work, we can still use ApiProperty on only those.

Two main benefits:

1. Less cluttered code
2. No more inconstencies (as you can see in my merge request I had to change about 20 things just for the dataset dtos)

[nitrosx]

@emigun your proposed solution look sensible.
I was wondering if you can create a PR with a POC for one of the simple sub-systems. So we can better discuss in one of the next meeting and start the implementation soon, if we can convince everybody and we do not run in any major set back.