英文:
How to avoid the 1GB $env:TEMP) in Azure Automation Powershell
问题
我在PowerShell Runbook中遇到了问题。我们扫描O365 AuditLogs并将结果写入磁盘，到目前为止，我们使用了$env:TEMP磁盘。
扫描后，我们将CSV文件上传到存储帐户。
但是现在我们的CSV文件变得大于1GB。
以下是代码的一部分：
$results = Search-UnifiedAuditLog -StartDate $currentStart -EndDate $currentEnd -SessionId $sessionID -SessionCommand ReturnLargeSet -ResultSize 5000
if (($results | Measure-Object).Count -ne 0) {
$outputFile = $outputPath + $recordType + "/Data_" + $currentStart.ToString("yyyyMMddHHmm") + "_" + $currentEnd.ToString("yyyyMMddHHmm") + "_" + $fileloopCount.ToString() + ".csv"
Check-Dir -dirName (Split-Path -Path $outputFile)
$results | export-csv -Path $outputFile -Append -NoTypeInformation
}
是否有其他工作方式？我考虑过追加和创建多个小文件，创建后上传并删除它们，但我想知道是否还有其他方法。
是否可以直接将Export-Csv用于Blob存储？
Kr，Harry
英文:
I'm having a problem in a Powershell Runbook. We scan the O365 AuditLogs and write the results to disk, up to this moment we userd the $env:TEMP disk.
After the scan we upload the csv files into a storage account.
But not our csv are getting bigger than 1GB.
THis s a part of the code ..
$results = Search-UnifiedAuditLog -StartDate $currentStart -EndDate $currentEnd -SessionId $sessionID -SessionCommand ReturnLargeSet -ResultSize 5000
if (($results | Measure-Object).Count -ne 0) {
$outputFile = $outputPath + $recordType + "/Data_" + $currentStart.ToString("yyyyMMddHHmm") + "_" + $currentEnd.ToString("yyyyMMddHHmm") + "_" + $fileloopCount.ToString() + ".csv"
Check-Dir -dirName (Split-Path -Path $outputFile)
$results | export-csv -Path $outputFile -Append -NoTypeInformation
Is there another way of working, I could not append and create multiple small files, upload them after creation and delete them, but I'm wondering if there would be another way.
Use Export-Csv directly to a blob-storage ?
Kr, Harry
答案1
得分: 0
您可以按照以下方法使用PowerShell上传1GB文件到存储账户。
你无法从Azure Runbook访问本地路径中的文件，查看此链接以获取更多详情。
我还尝试从本地计算机上传1GB文件到Azure存储，结果如下所示。
直接使用Export-Csv将数据导出到Blob存储？
PowerShell中的Export-Csv命令用于将数据导出到CSV文件，而不是直接导出到Azure中的Blob存储。
您可以使用以下PowerShell脚本将文件从本地上传到Azure存储账户。
# 连接到Azure
Connect-AzAccount -Identity
# 设置存储账户名和容器名
$storageAccountName = "emobro"
$containerName = "rithwik"
$storageAccountKey = "StorageAccountKey"
# 获取存储账户上下文
$storageContext = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
# 设置要上传的CSV文件的本地路径和文件名
$localFilePath = "C:\Users\v-vallepuv\Downloads\samplefile.csv"
$blobName = "sample.csv"
# 将CSV文件上传到Azure Blob存储
Set-AzStorageBlobContent -Context $storageContext -Container $containerName -File $localFilePath -Blob $blobName
运行上述PowerShell代码后，文件成功上传到Azure存储账户，如下所示。
英文:
You can follow below approach to upload 1GB file to storage account using powershell.
You cannot access the file located in local path from Azure Runbook, follow the Stack Link for more details by @Pankaj Rawat
I also tried to upload 1GB file from local machine to Azure storage using Azure Runbook, getting below result.
> Use Export-Csv directly to a blob-storage ?
Export-Csv command in PowerShell is used to export data to a CSV file, not directly to a blob storage in Azure.
You can use below powershell script to upload file from local to Azure Storage account.
#Authenticate to Azure
Connect-AzAccount -Identity
Set the storage account name and container name
$storageAccountName = "emobro"
$containerName = "rithwik"
$storageAccountKey ="StoarageaccountKey"
#Get the storage account context
$storageContext = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
Set the file path and name of the CSV file to upload
$localFilePath = "C:\Users\v-vallepuv\Downloads\samplefile.csv"
$blobName = "sample.csv"
Upload the CSV file to Azure Blob Storage
Set-AzStorageBlobContent -Context $storageContext -Container $containerName -File $localFilePath -Blob $blobName
Once ran the above powershell code, file got uploaded to Azure storage account successfully as below.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论