英文:
Spring SFTP : Unable to rename .writing file
问题
我正在使用Spring SFTP集成来传输文件,但很多次我都遇到了这个错误。看起来有两个线程尝试传输同一个文件并且彼此冲突。
>2020-08-03 08:31:55,766 INF [task-scheduler-8 ] o.s.i.ftp.session.FtpSession - 文件已成功从以下位置传输:./abc.ext.200803
<br>
>2020-08-03 08:31:55,849 INF [task-scheduler-7 ] o.s.i.ftp.session.FtpSession - 文件已成功从以下位置传输:./abc.ext.200803<br>
>2020-08-03 08:31:55,850 INF [task-scheduler-7 ] .s.i.f.i.FtpInboundFileSynchronizer - 在删除后,无法将'/local/download/abc.ext.200803.writing' 重命名为本地文件'/local/download/abc.ext.200803'。本地文件可能在其他进程中忙碌。<br>
是否有办法让这两个线程不相互干扰?
我正在使用以下代码 -
@Bean
public SftpInboundFileSynchronizer ftpInboundFileSynchronizer() {
isFTPSessionOK();
SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory());
fileSynchronizer.setPreserveTimestamp(true);
fileSynchronizer.setRemoteDirectory(remoteDirectory);
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));
return fileSynchronizer;
}
private boolean isFTPSessionOK() {
try {
SessionFactory<LsEntry> ftpSessionFactory = sftpSessionFactory();
boolean open = ftpSessionFactory.getSession().isOpen();
LOG.info("FTPSession is good ? " + open);
return open;
} catch (Exception e) {
LOG.error("FTPSession is not good because of error : " + e);
}
return false;
}
@Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory sf = new DefaultSftpSessionFactory();
sf.setHost(server);
sf.setPort(port);
sf.setUser(username);
sf.setPassword(password);
sf.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(sf);
}
@Bean
@InboundChannelAdapter(channel = "sftpChannel", poller = @Poller(fixedDelay = "${${project.name}.ftp.poller.delay:600000}", maxMessagesPerPoll = "1"))
public MessageSource<File> ftpMessageSource() {
SftpInboundFileSynchronizingMessageSource source = new SftpInboundFileSynchronizingMessageSource(ftpInboundFileSynchronizer());
source.setLocalDirectory(new File(localFtpDirectory));
source.setAutoCreateLocalDirectory(true);
return source;
}
@Bean
@ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler ftpHandler() {
return new MessageHandler() {
@Override
public void handleMessage(Message<?> message) throws MessagingException {
LOG.info("File '{}' is ready for reading after SFTP", message.getPayload());
}
};
}
英文:
I am using Spring SFTP integration to transfer the file and many time I got this error. It seems two threads are trying to transfer same file and conflict with each other
>2020-08-03 08:31:55,766 INF [task-scheduler-8 ] o.s.i.ftp.session.FtpSession - File has been successfully transferred from: ./abc.ext.200803
<br>
>2020-08-03 08:31:55,849 INF [task-scheduler-7 ] o.s.i.ftp.session.FtpSession - File has been successfully transferred from: ./abc.ext.200803<br>
>2020-08-03 08:31:55,850 INF [task-scheduler-7 ] .s.i.f.i.FtpInboundFileSynchronizer - Cannot rename '/local/download/abc.ext.200803.writing' to local file '/local/download/abc.ext.200803' after deleting. The local file may be busy in some other process.<br>
Is there a way so both threads should not interfere with each other?
I am using following code -
@Bean
public SftpInboundFileSynchronizer ftpInboundFileSynchronizer() {
isFTPSessionOK();
SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory());
fileSynchronizer.setPreserveTimestamp(true);
fileSynchronizer.setRemoteDirectory(remoteDirectory);
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));
return fileSynchronizer;
}
private boolean isFTPSessionOK() {
try {
SessionFactory<LsEntry> ftpSessionFactory = sftpSessionFactory();
boolean open = ftpSessionFactory.getSession().isOpen();
LOG.info("FTPSession is good ? " + open);
return open;
} catch (Exception e) {
LOG.error("FTPSession is not good because of error : " + e);
}
return false;
}
@Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory sf = new DefaultSftpSessionFactory();
sf.setHost(server);
sf.setPort(port);
sf.setUser(username);
sf.setPassword(password);
sf.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(sf);
}
@Bean
@InboundChannelAdapter(channel = "sftpChannel", poller = @Poller(fixedDelay = "${${project.name}.ftp.poller.delay:600000}", maxMessagesPerPoll = "1"))
public MessageSource<File> ftpMessageSource() {
SftpInboundFileSynchronizingMessageSource source = new SftpInboundFileSynchronizingMessageSource(ftpInboundFileSynchronizer());
source.setLocalDirectory(new File(localFtpDirectory));
source.setAutoCreateLocalDirectory(true);
return source;
}
@Bean
@ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler ftpHandler() {
return new MessageHandler() {
@Override
public void handleMessage(Message<?> message) throws MessagingException {
LOG.info("File '{}' is ready for reading after SFTP", message.getPayload());
}
};
}
答案1
得分: 1
以下是要翻译的内容:
你只有这个用于过滤的内容:
fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));
但是怎么样才能防止后续轮询中的重复项呢?
请查看AcceptOnceFileListFilter
。并且与SFTPLastModifiedFileFilter
一起使用ChainFileListFilter
。
请查阅文档以获取更多信息:
https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#sftp-inbound
https://docs.spring.io/spring-integration/docs/current/reference/html/file.html#file-reading
英文:
You have only this for filtering:
fileSynchronizer.setFilter(new SFTPLastModifiedFileFilter(remoteFileFilter));
but what about a filter which is going to prevent duplicates on subsequent poll?
See AcceptOnceFileListFilter
. And together with that SFTPLastModifiedFileFilter
you should use a ChainFileListFilter
.
See docs for more info:
https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#sftp-inbound
https://docs.spring.io/spring-integration/docs/current/reference/html/file.html#file-reading
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论