Azure阻止列表包含2个已提交的块,但结果blob仅显示第一个块。

huangapple go评论80阅读模式
英文:

Azure blocklist contains 2 committed blocks but the result blob only shows the first block

问题

我正在使用 Azure 的 REST API 来执行 "put block" 和 "put block list" 操作,这些操作都成功完成了。我使用以下 URL 的 "get" 方法来验证:https://accountname.blob.core.windows.net/container/myblobname?comp=blocklist
结果如下:

<?xml version="1.0" encoding="utf-8"?>
<BlockList>
    <CommittedBlocks>
        <Block>
            <Name>MjAyMy0wNy0xM1QxNTo0Mjo0MS4xM</Name>
            <Size>1048057</Size>
        </Block>
        <Block>
            <Name>MjAyMy0wNy0xM1QxNTo0MzoxMC44M</Name>
            <Size>460893</Size>
        </Block>
    </CommittedBlocks>
</BlockList>

这两个块与一个视频文件相关。我的问题是,当我使用该 URL (https://accountname.blob.core.windows.net/container/myblobname) 获取整个视频时,它只返回第一个块,第二个块丢失了。问题出在哪里?
请求详细信息请见以下代码:

Azure阻止列表包含2个已提交的块,但结果blob仅显示第一个块。

更新:以下是我的代码:

export const uploadLongFile = async (param) => {
  // ...(你的代码)
};

const stageBlock = async (blockId, chunk, url, length) => {
  // ...(你的代码)
};

const commitBlockList = async (blockIds, length, url) => {
  // ...(你的代码)
};

如果你有其他问题或需要更多帮助,请告诉我。

英文:

I am using rest api of azure tu put block and also put block list, the operations finished successfully. I checked this with get method of this url https://accountname.blob.core.windows.net/container/myblobname?comp=blocklist
the result is:

&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;
&lt;BlockList&gt;
    &lt;CommittedBlocks&gt;
        &lt;Block&gt;
            &lt;Name&gt;MjAyMy0wNy0xM1QxNTo0Mjo0MS4xM&lt;/Name&gt;
            &lt;Size&gt;1048057&lt;/Size&gt;
        &lt;/Block&gt;
        &lt;Block&gt;
            &lt;Name&gt;MjAyMy0wNy0xM1QxNTo0MzoxMC44M&lt;/Name&gt;
            &lt;Size&gt;460893&lt;/Size&gt;
        &lt;/Block&gt;
    &lt;/CommittedBlocks&gt;
&lt;/BlockList&gt;

These 2 blocks are related to a video file. my problem is that when I use the url (https://accountname.blob.core.windows.net/container/myblobname) to get whole video it just return first block so the second one is missing. where is the problem?
request details
Azure阻止列表包含2个已提交的块,但结果blob仅显示第一个块。

Updated: here is my code

export const uploadLongFile = async (param: any) =&gt; {
  const {path, filename, mime} = param;
  const blockSize = 1 * 1024 * 1024; // 1MB
  const fileSize = param?.size;
  const blockCount = Math.ceil(fileSize / blockSize);
  const sasContainerUri =  Config.CONTAINER_URI;
  const customBlobName = Math.random().toString(16).slice(2);
  const fileExtension = (filename ?? path).split(&#39;.&#39;).pop();
  const newFileName = `${customBlobName}.${fileExtension}`;
  const container = &#39;images&#39;;
  const assetPath = `${sasContainerUri}/${container}/${newFileName}`;
  const sasToken = Config.SAS_TOKEN; // you may need to play with other html verbs in this string e.g., `sp`, `ss` e.t.c.
  const blockIds: any[] = [];
  const videoChunks: any[] = [];
  const timeInterval = param?.duration / 1000 / blockCount;
  const chunks = await TrimWithIntervalVideo(path, timeInterval);// Trim video into blockCount chunks
  let totalSize = 0;
  for (let i = 0; i &lt; blockCount; i++) {
    let dataString = encodeURIComponent(encode(new Date().toISOString()));
    blockIds.push(dataString);
    let info = await ReactNativeBlobUtil.fs.stat(videoChunks[i]);
    let chunkPath =
      i &lt; 10
        ? `${chunks?.url}00${i}.${chunks?.ext}`
        : i &lt; 100
        ? `${chunks?.url}0${i}.${chunks?.ext}`
        : `${chunks?.url}${i}.${chunks?.ext}`;
    videoChunks.push(chunkPath);
    totalSize += info.size;
    await stageBlock(
      blockIds[i],
      videoChunks[i],
      `${assetPath}?${sasToken}`,
      info?.size,
    );
    if (i === blockCount - 1) {
      commitBlockList(blockIds, totalSize, `${assetPath}?${sasToken}`);
    }
  }
};
const stageBlock = async (
  blockId?: any,
  chunk?: any,
  url?: any,
  length?: any,
) =&gt; {
  const res = await ReactNativeBlobUtil.fetch(
    &#39;PUT&#39;,
    `${url}&amp;comp=block&amp;blockid=${blockId}`,
    {
      &#39;x-ms-date&#39;: new Date().toUTCString(),
      &#39;content-length&#39;: `${length}`,
    },
// Change BASE64 encoded data to a file path with prefix `ReactNativeBlobUtil-file://`.
// Or simply wrap the file path with ReactNativeBlobUtil.wrap(
    ReactNativeBlobUtil.wrap(chunk),
  ).catch(error =&gt; console.log({error}));
  console.log({res});
};
const commitBlockList = async (blockIds: any, length: any, url: any) =&gt; {
  let blockList: any[] = blockIds?.map(
    (value: string) =&gt; `&lt;Latest&gt;${value}&lt;/Latest&gt;`,
  );
  let ff = blockList
    .toString()
    .replace(&#39;[&#39;, &#39;&#39;)
    .replace(&#39;]&#39;, &#39;&#39;)
    .replace(&#39;,&#39;, &#39;\n&#39;);
  console.log({url}, {blockList}, {ff});
  const res2 = await ReactNativeBlobUtil.fetch(
    &#39;PUT&#39;,
    `${url}&amp;comp=blocklist`,
    {
      &#39;x-ms-date&#39;: new Date().toUTCString(),
      &#39;content-length&#39;: `${length}`,
      &#39;content-type&#39;: &#39;text/plain; charset=UTF-8&#39;,
    },
    `&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;
    &lt;BlockList&gt;
      ${ff}
    &lt;/BlockList&gt;`,
  )
    .progress((re, t) =&gt; console.log({re}, {t}))
    .catch(error =&gt; console.log({error}));
  console.log({res2});
};

答案1

得分: 0

感谢 @GauravMantri。我更改了提取块的方法,现在它运行良好。问题中的代码为每个块设置视频类型(例如chunk1.mp4),似乎Azure存储无法合并这些文件。因此,我将每个块存储为无类型,并更改了代码,现在它可以工作。

export const uploadLargeFile = async (param: any) => {
  const { path, filename, mime } = param;
  const blockSize = 1 * 1024 * 1024; // 1MB
  const fileSize = param?.size;
  const blockCount = Math.ceil(fileSize / blockSize);
  const sasContainerUri = Config.sasContainerUri;
  const customBlobName = Math.random().toString(16).slice(2);
  const fileExtension = (filename ?? path).split('.').pop();
  const newFileName = `${customBlobName}.${fileExtension}`;
  const container = 'images';
  const assetPath = `${sasContainerUri}/${container}/${newFileName}`;
  const sasToken = Config.SAS_TOKEN; // you may need to play with other html verbs in this string e.g., `sp`, `ss` e.t.c.
  const blockIds: any[] = [];
  let totalSize = 0;
  for (let i = 0; i < blockCount; i++) {
    const blockId = btoa('block-' + i.toString().padStart(6, '0'));
    blockIds.push(blockId);
    const start = i * blockSize;
    const end = Math.min(start + blockSize, fileSize);
    let dest = `${ReactNativeBlobUtil.fs.dirs.CacheDir}/RIM3/d3_${i}`;
    let file = await ReactNativeBlobUtil.fs.slice(path, dest, start, end);
    let info = await ReactNativeBlobUtil.fs.stat(file);
    const stageResult = await stageBlock(
      blockId,
      file,
      `${assetPath}?${sasToken}`,
      info?.size,
      i,
      blockCount,
      param?.changeProgress,
    );
    if (!stageResult) {
      showErrorMessage('Error upload file');
      break;
    }
    totalSize += info.size;
    console.log({ i });
    if (i === blockCount - 1) {
      const isSuccess = await commitBlockList(
        blockIds,
        fileSize,
        `${assetPath}?${sasToken}`,
      );
      if (isSuccess) {
        return newFileName;
      } else {
        return undefined;
      }
    }
  }
};

希望这对你有帮助!

英文:

thanks to @GauravMantri . I change my method to extract chunks and it works fine. Code in the question set video type for each chunk (for example chunk1.mp4) it seems azure storage cannot merge these files. So I store each chunk without type and change code to this, now its working.

export const uploadLargeFile = async (param: any) =&gt; {
const {path, filename, mime} = param;
const blockSize = 1 * 1024 * 1024; // 1MB
const fileSize = param?.size;
const blockCount = Math.ceil(fileSize / blockSize);
const sasContainerUri = Config.sasContainerUri ;
const customBlobName = Math.random().toString(16).slice(2);
const fileExtension = (filename ?? path).split(&#39;.&#39;).pop();
const newFileName = `${customBlobName}.${fileExtension}`;
const container = &#39;images&#39;;
const assetPath = `${sasContainerUri}/${container}/${newFileName}`;
const sasToken = Config.SAS_TOKEN; // you may need to play with other html verbs in this string e.g., `sp`, `ss` e.t.c.
const blockIds: any[] = [];
let totalSize = 0;
for (let i = 0; i &lt; blockCount; i++) {
const blockId = btoa(&#39;block-&#39; + i.toString().padStart(6, &#39;0&#39;));
blockIds.push(blockId);
const start = i * blockSize;
const end = Math.min(start + blockSize, fileSize);
let dest = `${ReactNativeBlobUtil.fs.dirs.CacheDir}/RIM3/d3_${i}`;
let file = await ReactNativeBlobUtil.fs.slice(path, dest, start, end);
let info = await ReactNativeBlobUtil.fs.stat(file);
const stageResult = await stageBlock(
blockId,
file,
`${assetPath}?${sasToken}`,
info?.size,
i,
blockCount,
param?.changeProgress,
);
if (!stageResult) {
showErrorMessage(&#39;Error upload file&#39;);
break;
}
totalSize += info.size;
console.log({i});
if (i === blockCount - 1) {
const isSuccess = await commitBlockList(
blockIds,
fileSize,
`${assetPath}?${sasToken}`,
);
if (isSuccess) {
return newFileName;
} else {
return undefined;
}
}
}
};

huangapple
  • 本文由 发表于 2023年7月14日 00:19:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/76681487.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定