S3-backed java.io.File

Here's my use case for a S3-backed Java File

File videoFile = new File(OUTPUT_FOLDER + "channel" + channel + "-" + System.currentTimeMillis() + ".mp4");
AWTSequenceEncoder encoder = AWTSequenceEncoder.createSequenceEncoder(videoFile, Long.valueOf(FPS).intValue());
 for (BufferedImage image : bufferedImages) {
      encoder.encodeImage(image);
}

Above is just one use-case, I am wondering if there exists a S3-backed File that can be used as a snap-in replacement for java.io.File; in the example code above the idea is to replace the videoFile of instead of persisting to disk, it will persist directly to S3 or S3-compatible storage.

And which is the best place to inject it, through compile-time or run-time?

AFAIK, no such thing exists.

The best you could do would be to implement an S3-backed file system at the operating system level. (For example, using s3backer linux man page) If the file system had the right properties, applications that used java.io.File wouldn't know the difference.

Alternatively, use a custom Java 7+ FileSystem javadoc, but you will have to change your code to stop using File and start using the more modern APIs.

^{(Of course, if you were willing to dive in and modify the JVM at the native code level, anything would be possible. But then you are introducing a whole stack of other problems.)}

> And which is the best place to inject it, through compile-time or run-time?

That is not a viable option. The important parts of java.io.File are implemented in native code in the JVM and are not "plugable" or "injectable".

There isn't a snap-in S3 replacement for java.io.File per se but there is a FileSystemProvider SPI that can be used as a provider on the JVM class path that will allow you to do things like use java.nio.Channels and Paths to access S3 objects via the URIs as if they were files. Take a look at https://github.com/awslabs/aws-java-nio-spi-for-s3

There actually exists an S3File (or was) from https://packages.atlassian.com/maven/com/squeakysand/aws/squeakysand-aws-s3/

Upon researching, the author is a previous AWS Architect, the project now seems to be abandoned, but the license is under Apache so a good fork might be able to revive it.

As for answering the best place to inject the S3File, the answer is compile-time. However, it might be quite intrusive to the base source code as such:

S3File videoFile = new S3File(BUCKET_NAME, fileKey);
AWTSequenceEncoder encoder = AWTSequenceEncoder.createSequenceEncoder(videoFile, Long.valueOf(FPS).intValue());
 for (BufferedImage image : bufferedImages) {
      encoder.encodeImage(image);
}

Something like this would work.

In fact, it should work with the S3File having the exact same parameters as of java.io.File so it can be used as S3File videoFile = new S3File(fileName);

Where the bucket name and other connection is already in the constructor.