使用Spring Data R2DBC进行批量插入时,检索生成的ID。

huangapple go评论73阅读模式
英文:

Retrieve generated IDs when doing bulk insert using Spring Data R2DBC

问题

我有一个情景,我的表格有一个自动生成的 id 列,我需要批量将项目插入数据库并获取生成的 id。有没有办法实现这个?

这是我的表格:

CREATE TABLE test_table (
  `id` SERIAL NOT NULL,
  `name` VARCHAR(100) NOT NULL,
  `created_date` DATETIME NOT NULL,
  PRIMARY KEY (`id`)
);

为了将一系列项目保存到这个表格中,我正在使用以下代码:

String initialSql = "INSERT INTO test_table(`name`,`created_date`) VALUES ";

List<String> values =
    dummyEntities.stream()
        .map(dummyEntity -> "('" + dummyEntity.getName() + "','" +
            dummyEntity.getCreatedDate().atZoneSameInstant(ZoneId.of("UTC")).toLocalDateTime().toString() + "')")
        .collect(Collectors.toList());

String sqlToExecute =  initialSql + String.join(",", values);
client.execute(sqlToExecute)
         //.//接下来怎么办?

生成的 SQL 语句(来自 DEBUG 日志):

2020-09-15 18:59:32.613 DEBUG 44801 --- [actor-tcp-nio-1] o.s.d.r2dbc.core.DefaultDatabaseClient   : 执行 SQL 语句 [INSERT INTO test_table(`name`,`created_date`) VALUES ('Abhi57','2020-09-15T13:29:29.951964'),('Abhi92','2020-09-15T13:29:29.952023')]

我甚至尝试使用 ConnectionFactory,但仍然不清楚怎么做:

Mono.from(connectionFactory.create())
    .map(Connection::createBatch)
    .map(batch -> {
      dummyEntities.forEach(dummyEntity -> {
        String sql = String.format("INSERT INTO `test_table` (`name`,`created_date`) VALUES ('%s','%s');", dummyEntity.getName(),
            dummyEntity.getCreatedDate().atZoneSameInstant(ZoneId.of("UTC")).toLocalDateTime().toString());
        batch.add(sql);
      });
      return batch;
    })
    .flatMap(batch -> Mono.from(batch.execute()))
    //.//接下来怎么办?

供参考,dummyEntities 变量保存着一个 DummyEntity 对象的列表。而 DummyEntity 类的定义如下:

@Table("test_table")
public class DummyEntity implements Persistable<Long> {

  @Id
  @Column("id")
  private Long id;

  @Column("name")
  private String name;

  @Column("created_date")
  private OffsetDateTime createdDate;

  //Getter,Setter
  @Override
  public boolean isNew() {
    return id == null;
  }
}

所使用的依赖版本:2.3.2.RELEASE

implementation 'org.springframework.boot:spring-boot-starter-webflux'
implementation 'org.springframework.boot:spring-boot-starter-data-r2dbc'
implementation 'dev.miku:r2dbc-mysql:0.8.2.RELEASE'
英文:

I have a scenario where my table has an autogenerated id column and I need to bulk insert items into db and fetch the generated ids. Is there any way I can achieve that?

This is my table:

CREATE TABLE test_table (
  `id` SERIAL NOT NULL,
  `name` VARCHAR(100) NOT NULL,
  `created_date` DATETIME NOT NULL,
  PRIMARY KEY (`id`)
);

To save a list of items into this table, the code I am using:

String initialSql = &quot;INSERT INTO test_table(`name`,`created_date`) VALUES &quot;;

    List&lt;String&gt; values =
        dummyEntities.stream()
            .map(dummyEntity -&gt; &quot;(&#39;&quot; + dummyEntity.getName() + &quot;&#39;,&#39;&quot;
                + dummyEntity.getCreatedDate().atZoneSameInstant(ZoneId.of(&quot;UTC&quot;)).toLocalDateTime().toString() + &quot;&#39;)&quot;)
            .collect(Collectors.toList());

    String sqlToExecute =  initialSql + String.join(&quot;,&quot;, values);
    client.execute(sqlToExecute)
             .//Then what?

The generated SQL statement (from DEBUG Logs):

2020-09-15 18:59:32.613 DEBUG 44801 --- [actor-tcp-nio-1] o.s.d.r2dbc.core.DefaultDatabaseClient   : Executing SQL statement [INSERT INTO test_table(`name`,`created_date`) VALUES (&#39;Abhi57&#39;,&#39;2020-09-15T13:29:29.951964&#39;),(&#39;Abhi92&#39;,&#39;2020-09-15T13:29:29.952023&#39;)]

I even tried using ConnectionFactory, still no clue

    Mono.from(connectionFactory.create())
        .map(Connection::createBatch)
        .map(batch -&gt; {
          dummyEntities.forEach(dummyEntity -&gt; {
            String sql = String.format(&quot;INSERT INTO `test_table` (`name`,`created_date`) VALUES (&#39;%s&#39;,&#39;%s&#39;);&quot;, dummyEntity.getName(),
                dummyEntity.getCreatedDate().atZoneSameInstant(ZoneId.of(&quot;UTC&quot;)).toLocalDateTime().toString());
            batch.add(sql);
          });
          return batch;
        })
        .flatMap(batch -&gt; Mono.from(batch.execute()))
        .//Then what?

For reference, dummyEntities variable holds a list of DummyEntity objects. And the DummyEntity class looks like this:

@Table(&quot;test_table&quot;)
public class DummyEntity implements Persistable&lt;Long&gt; {

  @Id
  @Column(&quot;id&quot;)
  private Long id;

  @Column(&quot;name&quot;)
  private String name;

  @Column(&quot;created_date&quot;)
  private OffsetDateTime createdDate;

  //Getter,Setter
  @Override
  public boolean isNew() {
    return id == null;
  }
}

Dependencies used: 2.3.2.RELEASE

    implementation &#39;org.springframework.boot:spring-boot-starter-webflux&#39;
    implementation &#39;org.springframework.boot:spring-boot-starter-data-r2dbc&#39;
    implementation &#39;dev.miku:r2dbc-mysql:0.8.2.RELEASE&#39;

答案1

得分: 2

使用原始的ConnectionFacotory很容易获得生成的ID。

我尝试使用ConnectionFacotory来获取生成的ID,结果如预期般工作。

                    .thenMany(
                            Mono.from(conn)
                                    .flatMapMany(
                                            c -> c.createStatement(insertSql)
                                                    .returnGeneratedValues("id")
                                                    .execute()

                                    )
                    )
                    .flatMap(data -> Flux.from(data.map((row, rowMetadata) -> row.get("id"))))
                    .doOnNext(id -> log.info("generated id: {}", id))

完整的代码示例在这里

它在控制台打印类似这样的日志。

2020-09-19 10:43:30,815 INFO [main] com.example.demo.H2Tests$Sql:89 generated id: 1
2020-09-19 10:43:30,815 INFO [main] com.example.demo.H2Tests$Sql:89 generated id: 2

我认为在Spring框架5.3中的新DatabaseClient只是连接工厂的薄包装,并使用filter来获取生成的ID。

databaseClient.sql("INSERT INTO  posts (title, content, metadata) VALUES (:title, :content, :metadata)")
.filter((statement, executeFunction) -> statement.returnGeneratedValues("id").execute())

查看完整示例代码(但此示例仅检索单个ID)。

英文:

Using the original ConnectionFacotory is easy to get generated ids.

I have tried to use ConnectionFactory to get the generated ids, worked as expected.

                    .thenMany(
                            Mono.from(conn)
                                    .flatMapMany(
                                            c -&gt; c.createStatement(insertSql)
                                                    .returnGeneratedValues(&quot;id&quot;)
                                                    .execute()

                                    )
                    )
                    .flatMap(data -&gt; Flux.from(data.map((row, rowMetadata) -&gt; row.get(&quot;id&quot;))))
                    .doOnNext(id -&gt; log.info(&quot;generated id: {}&quot;, id))

The complete code example is here.

It prints logs in the console like this.

2020-09-19 10:43:30,815 INFO [main] com.example.demo.H2Tests$Sql:89 generated id: 1
2020-09-19 10:43:30,815 INFO [main] com.example.demo.H2Tests$Sql:89 generated id: 2

And I think the new DatabaseClient in Spring framework 5.3 is just a thin wrapper of the connectionfactories,and use a filter to get generated ids.

databaseClient.sql(&quot;INSERT INTO  posts (title, content, metadata) VALUES (:title, :content, :metadata)&quot;)
.filter((statement, executeFunction) -&gt; statement.returnGeneratedValues(&quot;id&quot;).execute())

Check the complete example codes(but this example only retrieve a single id).

答案2

得分: 2

迟来的回复,但由于我在 R2DBC PostgreSQL 上遇到了相同的问题,这里是一个可能的解决方案:

// -- 生成 100 个产品实体并将它们标记为 CREATE
// -- Product 实现了 Persistable<UUID>,isNew() 对于 CREATE 将返回 TRUE,对于 UPDATE 将返回 FALSE
List<Product> list = Stream.generate(() -> Product.builder()
    .articleNumber(UUID.randomUUID().toString().substring(0, 10))
    .name(UUID.randomUUID().toString())
    .images(Arrays.asList("1", "2"))
    .saveAction(SaveAction.CREATE)
    .build()).limit(100).collect(Collectors.toList());

// -- 创建 PostgresqlBatch 并返回 Result 流
Flux<? extends Result> flux = databaseClient.inConnectionMany(connection -> {
  Batch batch = connection.createBatch();

  list.forEach(p -> {
    batch.add("INSERT INTO product(\"id\",\"article_number\",\"name\") VALUES ('" + p.getId() + "','" + p.getArticleNumber() + "','" + p
        .getName() + "') RETURNING id, name, article_number");
  });

  return Flux.from(batch.execute());
});

// -- 将 Result 流转换为 Product 流
return flux.flatMap(result -> result.map((row, rowMetadata) -> Product.builder()
    .id(row.get("id", UUID.class))
    .name(row.get("name", String.class))
    .articleNumber(row.get("article_number", String.class))
    .build()));

秘密 是在每个插入语句中添加 RETURNING column1, column2, column3

英文:

Late response, but since i faced the same issue on R2DBC PostgreSQL, here is a possible solution:

// -- generate 100 Product entities and mark them as CREATE
// -- Product is implementing Persistable&lt;UUID&gt; and isNew() will return TRUE for CREATE and FALSE for UPDATE 
List&lt;Product&gt; list = Stream.generate(() -&gt; Product.builder()
.articleNumber(UUID.randomUUID().toString().substring(0, 10))
.name(UUID.randomUUID().toString())
.images(Arrays.asList(&quot;1&quot;, &quot;2&quot;))
.saveAction(SaveAction.CREATE)
.build()).limit(100).collect(Collectors.toList());
// -- create the PostgresqlBatch and return the Result flux
Flux&lt;? extends Result&gt; flux = databaseClient.inConnectionMany(connection -&gt; {
Batch batch = connection.createBatch();
list.forEach(p -&gt; {
batch.add(&quot;INSERT INTO product(\&quot;id\&quot;,\&quot;article_number\&quot;,\&quot;name\&quot;) VALUES (&#39;&quot; + p.getId() + &quot;&#39;,&#39;&quot; + p.getArticleNumber() + &quot;&#39;,&#39;&quot; + p
.getName() + &quot;&#39;) RETURNING id, name, article_number&quot;);
});
return Flux.from(batch.execute());
});
// -- transform the Result flux into a Product flux
return flux.flatMap(result -&gt; result.map((row, rowMetadata) -&gt; Product.builder()
.id(row.get(&quot;id&quot;, UUID.class))
.name(row.get(&quot;name&quot;, String.class))
.articleNumber(row.get(&quot;article_number&quot;, String.class))
.build()));

The secret is RETURNING column1, column2, column3 added to each insert statement.

huangapple
  • 本文由 发表于 2020年9月15日 21:49:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/63903351.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定