将Java的InputStream作为文件参数发送到Linux命令

huangapple go评论143阅读模式
英文:

Sending Java InputStream to Linux Commands as a File Parameter

问题

最近,我参与了Storlet项目,这是OpenStack Swift项目的中间件。我不打算谈论Storlet,但简而言之,Storlet在存储在Swift对象存储中的对象(文件)上运行Java代码。文件由Storlet读取,并以InputSream的形式发送到Java应用程序,这意味着我们不能直接访问文件。

以下是一个Storlet的示例代码,它将图像作为InputStream输入,并制作其缩略图。

// ... 之前的代码

public class ThumbnailStorlet implements IStorlet {
    @Override
    public void invoke(ArrayList<StorletInputStream> inputStreams,
        ArrayList<StorletOutputStream> outputStreams,
        Map<String, String> parameters, StorletLogger log)
        throws StorletException {
    // ... 之后的代码

        /*
         * Convert
         */
        log.emitLog("Converting");
        int newH = img.getHeight()/8;
        int newW = img.getWidth()/8;
        int type = img.getTransparency() == Transparency.OPAQUE ? BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB;
        BufferedImage thumbnailImage = new BufferedImage(newW, newH, type);
        Graphics2D g = thumbnailImage.createGraphics();
        g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
        g.drawImage(img, 0, 0, newW, newH, null);
        g.dispose();

        /*
         * Write
         */
        log.emitLog("Writing Output");
        try {
            ImageIO.write(thumbnailImage, "PNG", thumbnailOutputStream);
        } catch (Exception e) {
            log.emitLog("Failed to write image to out stream");
            throw new StorletException("Failed to write image to out stream " + e.getMessage());
        } finally {
            try {
                thumbnailOutputStream.close();
            } catch (IOException e) {
            }
        }

        try {
            thumbnailOutputStream.close();
        } catch (IOException e) {
        }

        log.emitLog("Done");

    }
}

// ... 之后的代码

现在,我想在我的Java代码中使用一些外部应用程序,比如GDAL,处理图像,类似于上面的代码(并不完全相同)。GDAL具有一些命令行命令。例如,以下命令:

gdal_translate -of JPEG -co QUALITY=50 input.tif output.jpg

input.tif已经存储在我的对象存储中,Storlet可以读取它并将其作为InputStream提供给我,同时我已经了解如何在Java中使用ProcessBuilder运行外部进程,但是假设我将input.tif作为InputStream接收,而不是文件。

接下来,由于存储不足(可能对象非常大,超过2GB),并且性能下降,我不想将InputStream写回本地存储,而是在我的Java代码中运行它。

在Java中是否有一种方法可以将InputStream作为文件参数传递给外部进程,而不必将其存储在磁盘上。

我在Ubuntu Docker上运行我的代码。

英文:

Recently I have got involved into Storlet project which is a middleware of OpenStack Swift Project. I do not intend to talk about Storlet but, in short Storlet running a java code on objects(files) that stored into swift object storage. Files read by the storlet and send to the java application in form of InputSream which means we don't direct access to files.

this is a sample code of a storlet which gets the image as an inputstream and make a thumbnail of it.

import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.io.InputStream;
import java.io.OutputStream;
import org.openstack.storlet.common.IStorlet;
import org.openstack.storlet.common.StorletException;
import org.openstack.storlet.common.StorletInputStream;
import org.openstack.storlet.common.StorletLogger;
import org.openstack.storlet.common.StorletObjectOutputStream;
import org.openstack.storlet.common.StorletContainerHandle;
import org.openstack.storlet.common.StorletOutputStream;
import org.openstack.storlet.common.StorletUtils;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.awt.Image;
import java.awt.Transparency;
import java.awt.Graphics2D;
import java.awt.RenderingHints;
public class ThumbnailStorlet implements IStorlet {
@Override
public void invoke(ArrayList&lt;StorletInputStream&gt; inputStreams,
ArrayList&lt;StorletOutputStream&gt; outputStreams,
Map&lt;String, String&gt; parameters, StorletLogger log)
throws StorletException {
log.emitLog(&quot;ThumbnailStorlet Invoked&quot;);
/*
* Get input stuff
*/
HashMap&lt;String, String&gt; object_md;
StorletInputStream storletInputStream = inputStreams.get(0);
InputStream thumbnailInputStream = storletInputStream.getStream();
object_md = storletInputStream.getMetadata();
/*
* Get output stuff
*/
StorletObjectOutputStream storletObjectOutputStream = (StorletObjectOutputStream)outputStreams.get(0);
OutputStream thumbnailOutputStream = storletObjectOutputStream.getStream();
/*
* Set the output metadata
*/
log.emitLog(&quot;Setting metadata&quot;);
storletObjectOutputStream.setMetadata(object_md);
/*
* Read Input to BufferedImage
*/
log.emitLog(&quot;Reading Input&quot;);
BufferedImage img = null;
try {
img = ImageIO.read(thumbnailInputStream);
} catch (Exception e) {
log.emitLog(&quot;Failed to read input stream to buffered image&quot;);
throw new StorletException(&quot;Failed to read input stream to buffered image &quot; + e.getMessage());
} finally {
try {
thumbnailInputStream.close();
} catch (IOException e) {
log.emitLog(&quot;Failed to close input stream&quot;);
}
}
try {
thumbnailInputStream.close();
} catch (IOException e) {
log.emitLog(&quot;Failed to close input stream&quot;);
}
/*
* Convert
*/
log.emitLog(&quot;Converting&quot;);
int newH = img.getHeight()/8;
int newW = img.getWidth()/8;
int type = img.getTransparency() == Transparency.OPAQUE ? BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB;
BufferedImage thumbnailImage = new BufferedImage(newW, newH, type);
Graphics2D g = thumbnailImage.createGraphics();
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
g.drawImage(img, 0, 0, newW, newH, null);
g.dispose();
/*
* Write
*/
log.emitLog(&quot;Writing Output&quot;);
try {
ImageIO.write(thumbnailImage, &quot;PNG&quot; , thumbnailOutputStream);
} catch (Exception e) {
log.emitLog(&quot;Failed to write image to out stream&quot;);
throw new StorletException(&quot;Failed to write image to out stream &quot; + e.getMessage());
} finally {
try {
thumbnailOutputStream.close();
} catch (IOException e) {
}
}
try {
thumbnailOutputStream.close();
} catch (IOException e) {
}
log.emitLog(&quot;Done&quot;);
}

}

Now ..
I want to using some external application which running on images such as GDAL inside my java code ,assume code like the code above(not exactly doing the same as above). GDLA has some cli commands. For example this command

 gdal_translate -of JPEG -co QUALITY=50  input.tif output.jpg

The input.tif has already stored into my object storage and storlet can read it and give it to me as inputstream, also I have some practice how to run external process inside java with java ProcessBuilder but, imagine I receive input.tif as InputStream not a file.

Next, I don't want to write back InputStream to local storage where my application running there because of lack of storage ( maybe the object are very large, more the 2GBs) and also degrading performance.

Is there any way in java to pass InputStream to external process as a file argument without storing it on disk.

I am running my code on Ubuntu Docker

答案1

得分: 1

我不认为你能够这样做,一个 File 需要被存储在 文件系统(无论是本地还是远程)中才能被读取。你可以尝试基于 ByteArrayInputStream 来进行读取过程,但是 GDAL 进程应该支持那种类型的输入:

GDAL 进程的支持情况请参考:

https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-arg-src_dataset

根据 GDAL 文档,似乎不太可能实现:

<src_dataset>
源数据集名称。可以是文件名,数据源的 URL,或者多数据集文件的子数据集名称。
英文:

I don´t think you can do that, a File needs to be stored in a File system (either local or remote) to be read. You could try to base a ByteArrayInputStream to the reading process but the GDAL process should support that type of input:

https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-arg-src_dataset

As per the GDAL documentation, it does not seem possible:

&lt;src_dataset&gt;
The source dataset name. It can be either file name, URL of data source or subdataset name for multi-dataset files.

huangapple
  • 本文由 发表于 2020年10月26日 15:51:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/64533210.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定