Spring OpenApi DTOs in GET requests

We use OpenAPI to generate the interface description for the services exposed in one of our server applications. We use that openapi.yaml then to generate client-sided libraries that will interact with these services for various languages. As the server backend uses the typical Maven + Java Spring Boot techstack we use springdoc-openapi-maven-plugin

<plugin>
    <groupId>org.springdoc</groupId>
    <artifactId>springdoc-openapi-maven-plugin</artifactId>
    <version>1.4</version>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>generate</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <apiDocsUrl>http://localhost:8822/v3/api-docs</apiDocsUrl>
        <outputFileName>openapi.json</outputFileName>
        <outputDir>${openapi.outputDir}</outputDir>
    </configuration>
</plugin>

to generate the interface description and openapi-generator-maven-plugin

<plugin>
    <groupId>org.openapitools</groupId>
    <artifactId>openapi-generator-maven-plugin</artifactId>
    <version>6.3.0</version>
    <executions>
        <execution>
            <id>spring-webclient</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <inputSpec>.../${openapi.outputDir}/openapi.json</inputSpec>
                <generatorName>java</generatorName>
                <library>webclient</library>
                <output>${openapi.outputDir}/spring-webclient</output>
                <configOptions>
                    <groupId>com.whatever</groupId>
                    <artifactId>web-api-client</artifactId>
                    <artifactVersion>${env.CLIENT_VERSION}</artifactVersion>
                    <apiPackage>com.whatever.api</apiPackage>
                    <modelPackage>com.whatever.dto</modelPackage>
                    <generateModelTests>false</generateModelTests>
                    <generateApiTests>false</generateApiTests>
                </configOptions>
            </configuration>
        </execution>
    </executions>
</plugin>

to generate the client library project for Java based clients.

This setup works in general but for one of our services which allows the management of various tickets we have a couple of business methods that operate on a provided search filter. This is implemented through a DTO that has roughly 30+ properties that can be specified.

@Data
@AllArgsConstructor
@NoArgsConstructor
@Schema
public class TicketSearchFilterDTO implements Serializable {

  @Schema(
          description = "..."
  )
  @JsonInclude(JsonInclude.Include.NON_NULL)
  private String startsWithInternalIdOrTitle;
  ...
}

Within a Spring typical RestController named TicketController one of the methods is now defining a GET endpoint for querying a list of tickets that match the given search filter. In Java Spring syntax this is implemented as such:

@GetMapping
public Page<Ticket> getTickets(
    @ParameterObject TicketSearchFilterDTO filterDto,
    @ParameterObject @PageableDefault(size = 25) Pageable pageable
) {
  log.info("Getting tickets, filter {}, pageable {}", filterDto, pageable);
  TicketSearchFilter filter = ticketSearchFilterDTOConverter.convert(filterDto);
  return ticketSearchRepository.getTickets(filter, pageable);
}

which results in a OpenAPI interface definition like

openapi: 3.0.1
info:
  title: OpenAPI definition
  version: '@env.CI_COMMIT_REF_NAME@'
servers:
  ...
paths:
  ...
  /tickets:
    get:
      operationId: getTickets
      parameters:
      - explode: true
        in: query
        name: startsWithInternalIdOrTitle
        required: false
        schema:
          description: ...
          type: string
        style: form
      ...
      - description: Zero-based page index (0..N)
        explode: true
        in: query
        name: page
        required: false
        schema:
          default: 0
          minimum: 0
          type: integer
        style: form
      - description: The size of the page to be returned
        explode: true
        in: query
        name: size
        required: false
        schema:
          default: 25
          minimum: 1
          type: integer
        style: form
      - description: "Sorting criteria in the format: property,(asc|desc). Default\
          \ sort order is ascending. Multiple sort criteria are supported."
        explode: true
        in: query
        name: sort
        required: false
        schema:
          items:
            type: string
          type: array
        style: form
      responses:
        "404":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/NotFoundError'
          description: Not Found
        "400":
          content:
            '*/*':
              schema:
                type: object
          description: Bad Request
        "409":
          content:
            '*/*':
              schema:
                type: object
          description: Conflict
        "200":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/PageTicket'
          description: OK
      tags:
      - ticket-controller
      x-accepts: '*/*'

When generating the Java client code for this definition I will have a TicketControllerApi class available that provides the mentioned getTickets method. But instead of providing a TicketSearchFilterDTO as input parameter it exposes all of the 30+ possible search parameters as argument to that method.

openApiClient.getTicketControllerApi().getTickets("abc123", ..., 0, 25, List.of());

where the last 3 arguments represents the Pageable parameters for page, size and sort order. As we have so many search parameters nulling out all of the not used search filters is a bit tedious if we i.e. only want to allow a search via the internal ID or the title of a ticket. When invoking this method we see that a request to /tickets?startsWithInternalIdOrTitle=abc123&page=0&size=25 is made, which is picked up by our Spring ticket controller as intended. The client will only add properties to the URI that were specified and are therefore non-null values.

In an attempt to make the client code a bit less painful I stumbled on this Github issue where a user is using

@Parameter(
    explode = Explode.TRUE, 
    in = ParameterIn.QUERY, 
    content = @Content(
        schema = @Schema(implementation = TicketSearchFilterDTO.class, 
        ref = "#/components/schemas/TicketSearchFilterDTO")
    )
) final TicketSearchFilterDTO filterDto

instead of

@ParameterObject final TicketSearchFilterDTO filterDto

as argument definition for the getTickets(...) method. This will change the OpenAPI definition to

openapi: 3.0.1
info:
  title: OpenAPI definition
  version: '@env.CI_COMMIT_REF_NAME@'
servers:
  ...
paths:
  ...
  /tickets:
    get:
      operationId: getTickets
      parameters:
      - content:
          '*/*':
            schema:
              $ref: '#/components/schemas/TicketSearchFilterDTO'
        in: query
        name: filterDto
        required: true
      - description: Zero-based page index (0..N)
        explode: true
        in: query
        name: page
        required: false
        schema:
          default: 0
          minimum: 0
          type: integer
        style: form
      - description: The size of the page to be returned
        explode: true
        in: query
        name: size
        required: false
        schema:
          default: 25
          minimum: 1
          type: integer
        style: form
      - description: "Sorting criteria in the format: property,(asc|desc). Default\
          \ sort order is ascending. Multiple sort criteria are supported."
        explode: true
        in: query
        name: sort
        required: false
        schema:
          items:
            type: string
          type: array
        style: form
      responses:
        "404":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/NotFoundError'
          description: Not Found
        "400":
          content:
            '*/*':
              schema:
                type: object
          description: Bad Request
        "409":
          content:
            '*/*':
              schema:
                type: object
          description: Conflict
        "200":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/PageTicket'
          description: OK
      tags:
      - ticket-controller
      x-accepts: '*/*'
  ...
components:
  ...
  schemas:
    ...
    TicketSearchFilterDTO:
      properties:
        startsWithInternalIdOrTitle:
          description: ...
          type: string
        ...
      type: object
    ...

When generating the Java client stuff the actual usage now changes to

TicketSearchFilterDTO searchFilter = new TicketSearchFilterDTO();
searchFilter.setStartsWithInternalIdOrTitle("abc123");
openApiClient.getTicketControllerApi().getTickets(searchFilter, 0, 25, List.of());

which actually matches what we aim for, however, when sending a request with this implementation the OpenAPI client will serialize the DTO as is into the URI instead of the DTO's parameters when not null:

/tickets?filterDto=class%20TicketSearchFilterDTO%20%7B%0A%20%20%20%20startsWithInternalIdOrTitle%3A%20abc123%0A%20%20%20%20...&page=0&size=15

which our Spring backend controller can't handle and therefore will return a DTO that is actually null and therefore will result in the first 25 entries being returned. The generated client Java code does look like this:

public Mono<PageTicket> getTickets(TicketSearchFilterDTO filterDto, Integer page, Integer size, List<String> sort) throws WebClientResponseException {
    ParameterizedTypeReference<PageTicket> localVarReturnType = new ParameterizedTypeReference<PageTicket>() {};
    return getTicketsRequestCreation(filterDto, page, size, sort).bodyToMono(localVarReturnType);
}

private ResponseSpec getTicketsRequestCreation(TicketSearchFilterDTO filterDto, Integer page, Integer size, List<String> sort) throws WebClientResponseException {
    Object postBody = null;
    // verify the required parameter 'filterDto' is set
    if (filterDto == null) {
        throw new WebClientResponseException("Missing the required parameter 'filterDto' when calling getTickets", HttpStatus.BAD_REQUEST.value(), HttpStatus.BAD_REQUEST.getReasonPhrase(), null, null, null);
    }
    // create path and map variables
    final Map<String, Object> pathParams = new HashMap<String, Object>();

    final MultiValueMap<String, String> queryParams = new LinkedMultiValueMap<String, String>();
    final HttpHeaders headerParams = new HttpHeaders();
    final MultiValueMap<String, String> cookieParams = new LinkedMultiValueMap<String, String>();
    final MultiValueMap<String, Object> formParams = new LinkedMultiValueMap<String, Object>();

    queryParams.putAll(apiClient.parameterToMultiValueMap(null, "filterDto", filterDto));
    queryParams.putAll(apiClient.parameterToMultiValueMap(null, "page", page));
    queryParams.putAll(apiClient.parameterToMultiValueMap(null, "size", size));
    queryParams.putAll(apiClient.parameterToMultiValueMap(ApiClient.CollectionFormat.valueOf("multi".toUpperCase(Locale.ROOT)), "sort", sort));

    final String[] localVarAccepts = { 
        "*/*"
    };
    final List<MediaType> localVarAccept = apiClient.selectHeaderAccept(localVarAccepts);
    final String[] localVarContentTypes = { };
    final MediaType localVarContentType = apiClient.selectHeaderContentType(localVarContentTypes);

    String[] localVarAuthNames = new String[] {  };

    ParameterizedTypeReference<PageTicket> localVarReturnType = new ParameterizedTypeReference<PageTicket>() {};
    return apiClient.invokeAPI("/tickets", HttpMethod.GET, pathParams, queryParams, postBody, headerParams, cookieParams, formParams, localVarAccept, localVarContentType, localVarAuthNames, localVarReturnType);
}

and so the core issue should be contained in the way the filterDto parameter is added: queryParams.putAll(apiClient.parameterToMultiValueMap(null, "filterDto", filterDto));.

While manually customizing that code can surely solve this issue this DTO is just one of many and we have plenty more controllers that operate on the same premise. Also, manually checking which properties a DTO defines involves reflection which might not be needed as the definition of the TicketSearchFilterDTO should be there already in the openapi.yaml definition.

Not that for PUT or POST requests where the DTO is added with @RequestBody @Valid annotations, OpenAPI is perfectly fine in generating method that expose DTOs to clients directly when Java code is generated therefore. Also the response is picked up by the Spring controller this way without much issues here. So the problem here only exists for GET requests where the properties of the DTO should be encoded into the URI when performing the actual request.

As this shouldn't be a to uncommon case in practice I wonder if there is a way to generate client code that allows DTOs in GET requests being added to the URI called when a parameter is non-null? Or is there a better alternative to allow the usage of DTOs within GET requests via generated OpenAPI client code out of the box?

After investigating the issue at hand further and checking against the Swagger serialization documentation the generation of the openapi.yaml for the given controller method

  @GetMapping
  public Page<Ticket> getTickets(
    @Parameter(explode = Explode.TRUE,
            in = ParameterIn.QUERY,
            style = ParameterStyle.FORM
    ) final TicketSearchFilterDTO filterDto,
    @ParameterObject @PageableDefault(size = 25) Pageable pageable
  ) {
    log.info("Getting tickets, filter {}, pageable {}", filterDto, pageable);
    TicketSearchFilter filter = ticketSearchFilterDTOConverter.convert(filterDto);
    return ticketSearchRepository.getTickets(filter, pageable);
  }

which results in an openapi.yaml output like

openapi: 3.0.1
info:
  title: OpenAPI definition
  version: '@env.CI_COMMIT_REF_NAME@'
servers:
  ...
paths:
  ...
  /tickets:
    get:
      operationId: getTickets
      parameters:
      - explode: true
        in: query
        name: filterDto
        required: true
        schema:
          $ref: '#/components/schemas/TicketSearchFilterDTO'
        style: form
      - description: Zero-based page index (0..N)
        explode: true
        in: query
        name: page
        required: false
        schema:
          default: 0
          minimum: 0
          type: integer
        style: form
      - description: The size of the page to be returned
        explode: true
        in: query
        name: size
        required: false
        schema:
          default: 25
          minimum: 1
          type: integer
        style: form
      - description: "Sorting criteria in the format: property,(asc|desc). Default\
          \ sort order is ascending. Multiple sort criteria are supported."
        explode: true
        in: query
        name: sort
        required: false
        schema:
          items:
            type: string
          type: array
        style: form
      responses:
        "404":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/NotFoundError'
          description: Not Found
        "400":
          content:
            '*/*':
              schema:
                type: object
          description: Bad Request
        "409":
          content:
            '*/*':
              schema:
                type: object
          description: Conflict
        "200":
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/PageTicket'
          description: OK
      tags:
      - ticket-controller
      x-accepts: '*/*'
    ...
components:
  schemas:
    ...
    TicketSearchFilterDTO:
      properties:
        startsWithInternalIdOrTitle:
          description: ...
          type: string
        ...
      type: object
    ...

seems to be fine. This will result in mentioned client API that results in encapsulating the filter options within the desired DTO object. However, openapi-generator-maven-plugin seems to lack coverage of these cases somehow.

The generator uses Mustache for the client generation and the respective api.mustache for the input parameter will generate i.e. the following line as part of the RequestCreation sub method:

queryParams.putAll(apiClient.parameterToMultiValueMap(null, "filterDto", filterDto));

This line is in particular responsible for adding the given DTO to the query parameters that will be called by the HTTP client at the end. And this will unfortunately result in the whole DTO being added as fliterDto query parameter in a serialized from rather than its properties being added to the query parameters. That filterDto query parameter is though unknown by the Spring backend and thus will return the first 25 tickets if find regardless of the input.

The Maven plugin though allows to specify custom Mustache templates i.e. via the following lines added before the <configOptions> section:

<templateDirectory>
    ./open-api/templates
</templateDirectory>

Now it is just a matter of customizing api.mustache at the referenced line in the above-mentioned link with the following code:

        if ({{paramName}}.getClass().getSimpleName().endsWith("DTO")) {
            // iterate through DTOs properties
            Class<?> clazz = {{paramName}}.getClass();
            for (Field field : clazz.getDeclaredFields()) {
                if (Modifier.isPrivate(field.getModifiers()) && !Modifier.isStatic(field.getModifiers())) {
                    field.setAccessible(true);
                    String fieldName = field.getName();
                    try {
                        Object value = field.get({{paramName}});
                        queryParams.putAll(apiClient.parameterToMultiValueMap({{#collectionFormat}}ApiClient.CollectionFormat.valueOf("{{{.}}}".toUpperCase(Locale.ROOT)){{/collectionFormat}}{{^collectionFormat}}null{{/collectionFormat}}, fieldName, value));
                    } catch (Exception ex) {
                        ex.printStackTrace();
                    }
                }
            }
        } else {
            queryParams.putAll(apiClient.parameterToMultiValueMap({{#collectionFormat}}ApiClient.CollectionFormat.valueOf("{{{.}}}".toUpperCase(Locale.ROOT)){{/collectionFormat}}{{^collectionFormat}}null{{/collectionFormat}}, "{{baseName}}", {{paramName}}));
        }

and adding imports for

import java.lang.reflect.Field;
import java.lang.reflect.Modifier;

at the top of the Mustache file to modify the generated code so that any parameters which class name ends with DTO to replace the previous adding of the DTO object directly to the query parameters with adding all of its defined properties to the query parameter list.

While I'd prefer a solution that doesn't need reflection in first place and instead use the property definition from the component scheme directly, this at least gets the job done for now.