Testcontainers – Localstack – S3 – 虚拟主机寻址不起作用

huangapple go评论140阅读模式
英文:

Testcontainers - Localstack - S3 - Virtual Host Addressing Not Working

问题

尝试了最后几天,试图让 Testcontainers 与虚拟主机寻址一起工作,因为路径样式寻址已经过时。但是,我无论如何都无法让 AWS SDK V2 for Java 与 Testcontainers.LocalStackContainer 配合使用。

我的扩展 Testcontainer 如下所示:

public class LocalStackContainerExtended extends LocalStackContainer {
   public static final String BUCKET_NAME = "testBucket";
   // 由于在测试中没有运行任何 DNS 解析器,需要将存储桶包含在网络别名中
   public static final String LOCALSTACK_NETWORK_ALIAS = BUCKET_NAME + "." + "localstack";

   public LocalStackContainerExtended(DockerImageName dockerImageName) {
       super(dockerImageName);

       withServices(Service.S3);
       withNetworkAliases(LOCALSTACK_NETWORK_ALIAS);
   }

   @Override
   public void start() {
       super.start();
       try {
           execInContainer("awslocal", "s3api", "create-bucket", "--bucket", BUCKET_NAME);
       } catch (Exception e) {
           throw new RuntimeException();
    }
}

需要与 Localstack 通信的组件的 AWS SDK 客户端配置如下:

@Bean
// AwsConfigService 集合类的 AWS 覆盖值
public S3Client getS3Client(AwsConfigService awsConfigService) {
    S3ClientBuilder builder = S3Client.builder()
        .region(awsConfigService.getRegion());

    return awsConfigService.getOverrideEndpoint()
        .map(URI::create)
        .map(builder::endpointOverride)
        .orElse(builder).build();
}

用于组件的 Testcontainer 设置如下:

public class AppContainer extends GenericContainer<AppContainer>{
   ...
   public void setLocalstackConfig(LocalStackContainerExtended localstack) {
      withEnv("awsConfigService.region", localstack.getRegion());
      String overrideEndpoint = String.format("http://%s:%s", LOCALSTACK_NETWORK_ALIAS,"4566");
      withEnv("awsConfigService.override.endpoint", overrideEndpoint);
   }
   ...
}

但在运行客户端和 Testcontainer 之间的测试时,我总是收到以下错误:

指定的存储桶不存在(服务:S3,状态代码:404,请求 ID:7a62c49f-347e-4fc4-9331-6e8eEXAMPLE,扩展请求 ID:MzRISOwyjmnupEEE949DD2EAF7F477/JypPGXLh0OVFGcJaaO3KW/hRAqKOpIEEp

我错配置了什么或者缺少什么?

英文:

Been trying to get Testcontainers to work with virtual host addressing for the last couple of days as path style addressing is on the way out. But for the life of me I can't get the AWS SDK V2 for Java to work with Testcontainers.LocalStackContainer.

My extended Testcontainer looks like this:

public class LocalStackContainerExtended extends LocalStackContainer {
   public static final String BUCKET_NAME = &quot;testBucket&quot;;
   // Need to include bucket in network alias as we&#39;re not running any DNS resolver in test
   public static final String LOCALSTACK_NETWORK_ALIAS = BUCKET_NAME + &quot;.&quot; + &quot;localstack&quot;;

   public LocalStackContainerExtended(DockerImageName dockerImageName) {
       super(dockerImageName);

       withServices(Service.S3);
       withNetworkAliases(LOCALSTACK_NETWORK_ALIAS);
   }

   @Override
   public void start() {
       super.start();
       try {
           execInContainer(&quot;awslocal&quot;, &quot;s3api&quot;, &quot;create-bucket&quot;, &quot;--bucket&quot;, BUCKET_NAME);
       } catch (Exception e) {
           throw new RuntimeException();
    }
}

AWS SDK client config for the component that needs to talk with Localstack:

@Bean
// AwsConfigService collection class of aws override values
public S3Client getS3Client(AwsConfigService awsConfigService) {
    S3ClientBuilder builder = S3Client.builder()
        .region(awsConfigService.getRegion());

    return awsConfigService.getOverrideEndpoint()
        .map(URI::create)
        .map(builder::endpointOverride)
        .orElse(builder).build();
}

Testcontainer setup for the component:

public class AppContainer extends GenericContainer&lt;AppContainer&gt;{
   ...
   public void setLocalstackConfig(LocalStackContainerExtended localstack) {
      withEnv(&quot;awsConfigService.region&quot;, localstack.getRegion());
      String overrideEndpoint = String.format(&quot;http://%s:%s&quot;, LOCALSTACK_NETWORK_ALIAS,&quot;4566&quot;);
      withEnv(&quot;awsConfigService.override.endpoint&quot;, overrideEndpoint);
   }
   ...
}

But when running tests between the the client and Testcontainer I always get the following error:

> The specified bucket does not exist (Service: S3, Status Code: 404, Request ID: 7a62c49f-347e-4fc4-9331-6e8eEXAMPLE, Extended Request ID: MzRISOwyjmnupEEE949DD2EAF7F477/JypPGXLh0OVFGcJaaO3KW/hRAqKOpIEEp

What have I miss-configured or what is missing?

答案1

得分: 1

以下是您要翻译的代码部分:

public class LocalStackContainerExtended extends LocalStackContainer {
   public static final String BUCKET_NAME = "testBucket";
   // Need to include bucket in network alias as we're not running any DNS resolver in test
   public static final String LOCALSTACK_S3_NETWORK_ALIAS = "s3.localhost.localstack.cloud";
   // You need to also include an alias for the to-be-created bucket
   public static final String BUCKET_LOCALSTACK_S3_NETWORK_ALIAS = BUCKET_NAME + "." + LOCALSTACK_S3_NETWORK_ALIAS

   public LocalStackContainerExtended(DockerImageName dockerImageName) {
       super(dockerImageName);

       withServices(Service.S3);
       withNetworkAliases(LOCALSTACK_S3_NETWORK_ALIAS);
       withNetworkAliases(BUCKET_LOCALSTACK_S3_NETWORK_ALIAS);
   }

   @Override
   public void start() {
       super.start();
       try {
           execInContainer("awslocal", "s3api", "create-bucket", "--bucket", BUCKET_NAME);
       } catch (Exception e) {
           throw new RuntimeException();
       }
   }
}

public class AppContainer extends GenericContainer<AppContainer>{
   ...
   public void setLocalstackConfig(LocalStackContainerExtended localstack) {
      withEnv("awsConfigService.region", localstack.getRegion());
      String overrideEndpoint = String.format("http://%s:%s", LOCALSTACK_S3_NETWORK_ALIAS, "4566");
      withEnv("awsConfigService.override.endpoint", overrideEndpoint);
   }
   ...
}

希望这对您有所帮助。如果您有其他问题或需要更多帮助,请随时告诉我。

英文:

It might be that LocalStack is not recognising your incoming request as a Virtual Host addressed one.
Could try replacing the network alias with the followings?

public class LocalStackContainerExtended extends LocalStackContainer {
   public static final String BUCKET_NAME = &quot;testBucket&quot;;
   // Need to include bucket in network alias as we&#39;re not running any DNS resolver in test
   public static final String LOCALSTACK_S3_NETWORK_ALIAS = &quot;s3.localhost.localstack.cloud&quot;;
   // You need to also include an alias for the to-be-created bucket
   public static final String BUCKET_LOCALSTACK_S3_NETWORK_ALIAS = BUCKET_NAME + &quot;.&quot; + LOCALSTACK_S3_NETWORK_ALIAS

   public LocalStackContainerExtended(DockerImageName dockerImageName) {
       super(dockerImageName);

       withServices(Service.S3);
       withNetworkAliases(LOCALSTACK_S3_NETWORK_ALIAS);
       withNetworkAliases(BUCKET_LOCALSTACK_S3_NETWORK_ALIAS);
   }

   @Override
   public void start() {
       super.start();
       try {
           execInContainer(&quot;awslocal&quot;, &quot;s3api&quot;, &quot;create-bucket&quot;, &quot;--bucket&quot;, BUCKET_NAME);
       } catch (Exception e) {
           throw new RuntimeException();
    }
}

When receiving a request with this host, S3 should recognise the request and parse it properly. What is happening right now is that it does not recognise the bucket in your request, as it does not extract it from the host because it does not follow the pattern.

More informations about virtual host addressing in LocalStack: https://docs.localstack.cloud/user-guide/aws/s3/#getting-started

You then need to also configure the client to use the right endpoint. Your S3 client will take care of prefixing your endpoint with the bucket name when targeting S3, so you need to use LOCALSTACK_S3_NETWORK_ALIAS as the overrideEndpoint (but beware, only for the client targeting S3!). Most other clients can use the endpoint localstack:4566 pointing to your container.

public class AppContainer extends GenericContainer&lt;AppContainer&gt;{
   ...
   public void setLocalstackConfig(LocalStackContainerExtended localstack) {
      withEnv(&quot;awsConfigService.region&quot;, localstack.getRegion());
      String overrideEndpoint = String.format(&quot;http://%s:%s&quot;, LOCALSTACK_S3_NETWORK_ALIAS,&quot;4566&quot;);
      withEnv(&quot;awsConfigService.override.endpoint&quot;, overrideEndpoint);
   }
   ...
}

huangapple
  • 本文由 发表于 2023年3月7日 17:52:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/75660363.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定