英文:
Streaming file with camel and readLock=none
问题
我正在尝试使用Apache Camel来消费（流式传输）一个大的zip文件。流式传输应该在文件开始写入时就开始。以下是文件消费者的代码。
rest("/api/request/{Id}/")
.get()
.produces(MediaType.APPLICATION_OCTET_STREAM_VALUE)
.process(new FindFileName)
.pollEnrich().simple("file:" + outputDir + "?fileName=${property.filnavn}&noop=false&readLock=none&delete=true").timeout(pollTimeout)
Claus Ibsen建议使用readLock=none来获取流。
当我使用这个选项时，流会立即关闭，我只得到一个带有正确文件名的0字节文件。
我应该如何配置Camel的文件终端点以使用readLock=none并消费文件直到完成？
另外一个路由在写入该文件。
英文:
I am trying to consume (stream) a big zip file with Apache Camel. The streaming should begin as soon as the file is being written to. Below is the file consumer code.
rest("/api/request/{Id}/")
.get()
.produces(MediaType.APPLICATION_OCTET_STREAM_VALUE)
.process(new FindFileName)
.pollEnrich().simple("file:" + outputDir + "?fileName=${property.filnavn}&noop=false&readLock=none&delete=true").timeout(pollTimeout)
Claus Ibsen suggested using readLock=none to get the stream.
When I use the option the stream closes right away and I only get the 0 byte file with the correct filename.
How do I configure camel's file endpoint to use readLock=none and consume the file until it is completed?
A seperate route writes the file.
答案1
得分: 1
没有安全的方法可以确定第三方何时完成对文件的写入。您在这种情况下所做的是，在轮询增强中获取一个java.io.File，该文件可以被Camel转换为FileInputStream以进行读取。但是该流没有办法知道第三方何时完成对文件的写入。
因此，这实际上是一种不良的做法，读取正在进行写入的文件。
要知道何时完成文件的写入，第三方可能会使用以下策略：
- 写入第二个虚拟标记文件以表示已完成
- 写入第二个正在进行的虚拟文件以表示文件当前正在被写入，并在完成时删除此文件
- 使用临时名称写入文件，并在完成时重命名
- 在另一个文件夹中写入文件，并在完成时移动
- 监视文件的修改时间戳，如果在X时间后时间戳不变，则假定文件已完成写入
- 尝试重命名文件，并假设如果操作系统无法执行此操作，则第三方仍在向文件中写入
- 等等...
JDK的文件锁定API不适用于跨文件系统，并且通常很难用于获取文件锁定 - 它可能在同一个JVM内起作用，但在两个不同的系统之间却无法起作用。
英文:
There is no safe way to know when a file is completed written by a 3rd party. What you do there, is that you get a hold of a java.io.File in the poll enrich to the file. Which Camel can convert to a FileInputStream to read from. But that stream has no way of knowing when the 3rd party if finished writing the file.
There its really a bad practice to read files that are currently in progress of being written.
To know when a file is complete written then 3rd parties may use a strategy to
- write a 2nd dummy marker file to tell its finished
- write a 2nd in-progress dummy file to tell the file is currently being written and delete this file when its finished
- write the file using a temporary name and rename when done
- write the file in another folder and move when done
- monitor the file for modified timestamp and if the timestamp doesnt change after X period then assume its finished written
- attempt to rename the file and assuming if the OS fails doing this then the 3rd party is still writing to the file
- etc...
The JDK File Lock API does not work acrosss file systems and is generally not very useable to get file locks - it may work from within the same JVM, but not when its 2 different systems.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论