英文:
Stream content to Google Cloud Storage
问题
我想将一个大的 Set<Integer> 上传到Google Cloud Storage。我可以使用以下代码实现：
Blob result = storage.create(blobInfo, Joiner.on('\n').join(set).getBytes(UTF_8));
但这样会创建一个包含所有内容的中间字符串，可能会太大。我在一个示例链接中找到了使用 WriteChannel.write() 的示例：
Set<Integer> set = ...
String bucketName = "my-unique-bucket";
String blobName = "my-blob-name";
BlobId blobId = BlobId.of(bucketName, blobName);
byte[] content = Joiner.on('\n').join(set).getBytes(UTF_8);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
try (WriteChannel writer = storage.writer(blobInfo)) {
writer.write(ByteBuffer.wrap(content, 0, content.length));
} catch (IOException ex) {
// 处理异常
}
然而，如果这样做，整个 set 首先会被转换成字符串，然后再转换成 byte[]。字符串本身可能会太大。
是否有一个示例演示如何迭代遍历集合并将其转换为 ByteBuffer？或者我应该对集合的分块进行循环操作？
英文:
I would like to upload a large Set<Integer> to Google Cloud Storage. I can do that with:
Blob result = storage.create(blobInfo, Joiner.on('\n').join(set).getBytes(UTF_8));
But this will create an intermediate String with all the content that might be too large.
I found an example with WriteChannel.write():
Set<Integer> set = ...
String bucketName = "my-unique-bucket";
String blobName = "my-blob-name";
BlobId blobId = BlobId.of(bucketName, blobName);
byte[] content = Joiner.on('\n').join(set).getBytes(UTF_8);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
try (WriteChannel writer = storage.writer(blobInfo)) {
writer.write(ByteBuffer.wrap(content, 0, content.length));
} catch (IOException ex) {
// handle exception
}
However, if I do that, the entire set is converted to a String and then to byte[]. The String itself might be too big.
Is there an example how to iterate over the set and transform it to a ByteBuffer? or should I do a loop on chunks of the set?
答案1
得分: 1
最直接的方法是：
try (WriteChannel writer = storage.writer(blobInfo)) {
for(Integer val : set) {
String valLine = val.toString() + '\n';
writer.write(ByteBuffer.wrap(valLine.getBytes(UTF_8)));
}
}
需要注意的是，这并不是非常高效的方法。它会创建许多小的 ByteBuffer。你可以通过将数据写入一个较大的单一 ByteBuffer，并定期使用 writer.write 进行优化。
英文:
The most straightforward approach I could think of would be:
try (WriteChannel writer = storage.writer(blobInfo)) {
for(Integer val : set) {
String valLine = val.toString() + '\n';
writer.write(ByteBuffer.wrap(valLine.getBytes(UTF_8));
}
}
Mind you, this isn't very efficient. It creates a lot of small ByteBuffers. You could greatly improve on this by writing into a single larger ByteBuffer and periodically calling writer.write with it.
答案2
得分: 0
为了避免创建包含所有字节的中间字符串，您可以直接从文件中进行上传。您可以在各种编程语言中找到从文件上传的示例代码此处。
英文:
To avoid creating an intermediate String with all the bytes you can upload from a file. You can find example code to do an upload from a file in various languages here.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论