如何使用testcontainers启动一个clickhouse容器?

huangapple go评论81阅读模式
英文:

How to start a clickhouse container with testcontainers?

问题

我想在集成测试中使用testcontainers。我需要对clickhouse存储进行测试。

Docker镜像是yandex/clichouse-server

到目前为止,我的代码(大部分从testcontainers网站的官方redis示例导入)如下所示:

	ctx := context.Background()
	req := testcontainers.ContainerRequest{
		Image:        "yandex/clickhouse-server",
		ExposedPorts: []string{"9000/tcp"},
	}
	chContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	require.NoError(t, err, "创建clickhouse容器时出现意外错误")

	endpoint, err := chContainer.Endpoint(ctx, "")
	require.NoError(t, err)

在获取端点时,会抛出port not found错误,我不确定接下来该怎么做。

英文:

I'd like to use testcontainers for integration testing. I need to test against a clickhouse storage.

The docker image is yandex/clichouse-server

My code thus far (imported mostly from the official redis example on testcontainers website):

	ctx := context.Background()
	req := testcontainers.ContainerRequest{
		Image: "yandex/clickhouse-server",
		ExposedPorts: []string{"9000/tcp"},
	}
	chContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	require.NoError(t, err, "unexpected error while creating clickhouse container")

	endpoint, err := chContainer.Endpoint(ctx, "")
	require.NoError(t, err)

This throws an error port not found on getting the endpoint, and I'm not sure where to go from there.

答案1

得分: 2

你尝试过在Testcontainers Go中使用等待API吗?你可以在这里找到它们:https://github.com/testcontainers/testcontainers-go/tree/main/wait

使用这些API,你可以同时等待多个条件:

  • 日志条目
  • 端口准备就绪
  • SQL查询
  • HTTP请求
  • 在容器中运行程序后的退出代码

你可以在该代码库中找到有用的示例。例如,下面是一个等待日志条目的示例:

ctx := context.Background()
req := ContainerRequest{
    Image:        "docker.io/mysql:latest",
    ExposedPorts: []string{"3306/tcp", "33060/tcp"},
    Env: map[string]string{
        "MYSQL_ROOT_PASSWORD": "password",
        "MYSQL_DATABASE":      "database",
    },
    WaitingFor: wait.ForLog("test context timeout").WithStartupTimeout(1 * time.Second),
}
_, err := GenericContainer(ctx, GenericContainerRequest{
    ProviderType:     providerType,
    ContainerRequest: req,
    Started:          true,
})

编辑:下面是一个更详细的示例,包括使用HTTP请求的等待策略:

const (
    dbName       = "crazy"
    fakeUser     = "jondoe"
    fakePassword = "bond girl"
)

ctx := context.Background()

req := ContainerRequest{
    Image: "clickhouse/clickhouse-server",
    Env: map[string]string{
        "CLICKHOUSE_DB":       dbName,
        "CLICKHOUSE_USER":     fakeUser,
        "CLICKHOUSE_PASSWORD": fakePassword,
    },
    ExposedPorts: []string{
        "8123/tcp",
        "9000/tcp",
    },
    WaitingFor: wait.ForAll(
        wait.ForHTTP("/ping").WithPort("8123/tcp").WithStatusCodeMatcher(
            func(status int) bool {
                return status == http.StatusOK
            },
        ),
    ),
}

clickhouseContainer, err := GenericContainer(ctx, GenericContainerRequest{
    ContainerRequest: req,
    Started:          true,
})
if err != nil {
    t.Fatal(err)
}

defer clickhouseContainer.Terminate(ctx)
英文:

Have you tried using the wait APIs in Testcontainers Go? https://github.com/testcontainers/testcontainers-go/tree/main/wait

With them you'll be able to wait for multiple things (even at the same time):

  • a log entry
  • a port to be ready
  • a SQL query
  • an HTTP request
  • an exit code after running a program in the container

You can find useful examples in the repository. I.e., and example for a log entry:

ctx := context.Background()
	req := ContainerRequest{
		Image:        "docker.io/mysql:latest",
		ExposedPorts: []string{"3306/tcp", "33060/tcp"},
		Env: map[string]string{
			"MYSQL_ROOT_PASSWORD": "password",
			"MYSQL_DATABASE":      "database",
		},
		WaitingFor: wait.ForLog("test context timeout").WithStartupTimeout(1 * time.Second),
	}
	_, err := GenericContainer(ctx, GenericContainerRequest{
		ProviderType:     providerType,
		ContainerRequest: req,
		Started:          true,
	})

EDIT: a more elaborated example, including a wait strategy using HTTP requests would be:

const (
		dbName       = "crazy"
		fakeUser     = "jondoe"
		fakePassword = "bond girl"
	)

	ctx := context.Background()

	req := ContainerRequest{
		Image: "clickhouse/clickhouse-server",
		Env: map[string]string{
			"CLICKHOUSE_DB":       dbName,
			"CLICKHOUSE_USER":     fakeUser,
			"CLICKHOUSE_PASSWORD": fakePassword,
		},
		ExposedPorts: []string{
			"8123/tcp",
			"9000/tcp",
		},
		WaitingFor: wait.ForAll(
			wait.ForHTTP("/ping").WithPort("8123/tcp").WithStatusCodeMatcher(
				func(status int) bool {
					return status == http.StatusOK
				},
			),
		),
	}

	clickhouseContainer, err := GenericContainer(ctx, GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	if err != nil {
		t.Fatal(err)
	}

	defer clickhouseContainer.Terminate(ctx)

答案2

得分: 0

这是我在试验和错误之后得到的工作代码:

const (
	dbName       = "crazy"
	fakeUser     = "jondoe"
	fakePassword = "bond girl"
)

// NewTestClickhouseDB spins up a new clickhouse container database
func NewTestClickhouseDB(t *testing.T) *sql.DB {
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	t.Cleanup(cancel)
	req := testcontainers.ContainerRequest{
		Image: "clickhouse/clickhouse-server",
		Env: map[string]string{
			"CLICKHOUSE_DB":       dbName,
			"CLICKHOUSE_USER":     fakeUser,
			"CLICKHOUSE_PASSWORD": fakePassword,
		},

		ExposedPorts: []string{"9000/tcp"},
		WaitingFor:   wait.ForListeningPort("9000"),
	}
	chContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	require.NoError(t, err, "创建 clickhouse 容器时出现意外错误")

	p, err := chContainer.MappedPort(ctx, "9000")
	require.NoError(t, err, "在 clickhouse 容器上找不到映射的端口")

	addr := fmt.Sprintf("127.0.0.1:%d", p.Int())
	conn := clickhouse.OpenDB(&clickhouse.Options{
		Addr: []string{addr},
		Auth: clickhouse.Auth{
			Database: dbName,
			Username: fakeUser,
			Password: fakePassword,
		},
	})

	for {
		if ctx.Err() != nil {
			t.Fatalf("连接数据库失败,超时时间为10秒")
		}
		err := conn.Ping()
		if err != nil {
			time.Sleep(10 * time.Millisecond)
			continue
		}
		break
	}

	return conn
}

这会启动一个 clickhouse 容器,并在10秒后返回 sql.Db 或超时。

英文:

Here is what I got working after trial / errors:

const (
dbName       = "crazy"
fakeUser     = "jondoe"
fakePassword = "bond girl"
)
// NewTestClickhouseDB spins up a new clickhouse container database
func NewTestClickhouseDB(t *testing.T) *sql.DB {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
t.Cleanup(cancel)
req := testcontainers.ContainerRequest{
Image: "clickhouse/clickhouse-server",
Env: map[string]string{
"CLICKHOUSE_DB":       dbName,
"CLICKHOUSE_USER":     fakeUser,
"CLICKHOUSE_PASSWORD": fakePassword,
},
ExposedPorts: []string{"9000/tcp"},
WaitingFor:   wait.ForListeningPort("9000"),
}
chContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started:          true,
})
require.NoError(t, err, "unexpected error while creating clickhouse container")
p, err := chContainer.MappedPort(ctx, "9000")
require.NoError(t, err, "expected mapped port to be found on clickhouse container")
addr := fmt.Sprintf("127.0.0.1:%d", p.Int())
conn := clickhouse.OpenDB(&clickhouse.Options{
Addr: []string{addr},
Auth: clickhouse.Auth{
Database: dbName,
Username: fakeUser,
Password: fakePassword,
},
})
for {
if ctx.Err() != nil {
t.Fatalf("time/out: ping db failed for 10seconds")
}
err := conn.Ping()
if err != nil {
time.Sleep(10 * time.Millisecond)
continue
}
break
}
return conn
}

This spins up a clickhouse container and returns the sql.Db or t/o after 10 seconds.

huangapple
  • 本文由 发表于 2022年9月9日 01:11:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/73652833.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定