update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

huangapple go评论65阅读模式
英文:

NotImplementedError in update with AzureMachineLearningFileSystem

问题

I have a problem with upload on azure
https://learn.microsoft.com/pl-pl/azure/machine-learning/how-to-access-data-interactive?view=azureml-api-2&tabs=adls
I use sample

from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<workspace_name>/datastores/<datastorename>/paths/')

# you can specify recursive as False to upload a file
fs.upload(lpath='data/upload_files/crime-spring.csv', rpath='data/fsspec', recursive=False, **{'overwrite': 'MERGE_WITH_OVERWRITE'})

# you need to specify recursive as True to upload a folder
fs.upload(lpath='data/upload_folder/', rpath='data/fsspec_folder', recursive=True, **{'overwrite': 'MERGE_WITH_OVERWRITE'})

This is offical solution.

When i try

from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/'+auth_subscription_id+'/resourcegroups/'+auth_resource_group+'/workspaces/'+auth_workspace_name+'/datastores/azure_churn_lab_datastores/paths/')

# you can specify recursive as False to upload a file
fs.upload(lpath='/' + path + seq_name + '.csv', 
          rpath='/' + path_target)

I have error with NotImplementedError:
I havent information about this problem... I tried upgrade package azure sdk.

I want ask you - this is problem with implementation or azure?
I know about different way to upload with folder.

Second question
If you prepare data in python sdk2 and default path is

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

You upload to blob containers.

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。,

or leave in this path (File Share)? And why? Because I saw - few people save and upload update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

I use Visual Studio Code - connect to

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

and when I save a file

df.to_csv('/test.csv')

default path is File Share code

英文:

I have a problem with upload on azure
https://learn.microsoft.com/pl-pl/azure/machine-learning/how-to-access-data-interactive?view=azureml-api-2&amp;tabs=adls
I use sample

from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem(&#39;azureml://subscriptions/&lt;subid&gt;/resourcegroups/&lt;rgname&gt;/workspaces/&lt;workspace_name&gt;/datastores/&lt;datastorename&gt;/paths/&#39;)

# you can specify recursive as False to upload a file
fs.upload(lpath=&#39;data/upload_files/crime-spring.csv&#39;, rpath=&#39;data/fsspec&#39;, recursive=False, **{&#39;overwrite&#39;: &#39;MERGE_WITH_OVERWRITE&#39;})

# you need to specify recursive as True to upload a folder
fs.upload(lpath=&#39;data/upload_folder/&#39;, rpath=&#39;data/fsspec_folder&#39;, recursive=True, **{&#39;overwrite&#39;: &#39;MERGE_WITH_OVERWRITE&#39;})

This is offical solution.

When i try

from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem(&#39;azureml://subscriptions/&#39;+auth_subscription_id+&#39;/resourcegroups/&#39;+auth_resource_group+&#39;/workspaces/&#39;+auth_workspace_name+&#39;/datastores/azure_churn_lab_datastores/paths/&#39;)

# you can specify recursive as False to upload a file
fs.upload(lpath=&#39;/&#39; + path + seq_name + &#39;.csv&#39;, 
          rpath=&#39;/&#39; + path_target)

I have error with NotImplementedError:
I havent information about this problem... I tried upgrade package azure sdk.

I want ask you - this is problem with implementation or azure?
I know about different way to upload with folder.

Second question
If you prepare data in python sdk2 and default path is

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

You upload to blob containers.

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。,

or leave in this path (File Share)? And why? Because I saw - few people save and upload update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

I use Visual Studio Code - connect to

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

and when I save a file

df.to_csv(&#39;/test.csv&#39;)

default path is File Share code

答案1

得分: 1

NotImplementedError: I haven't information about this problem...

在使用 AzureMachineLearningFileSystem 类上传文件时出现 NotImplementedError 错误,这很可能是 azureml-sdk 和 **fsspec 包之间版本冲突的结果。为了检查是否解决了问题,请尝试将这两个包都更新到最新或相关版本。

我尝试使用相同的代码上传文件,因为你选择了默认目录(文件共享目录)。

代码:

from azureml.fsspec import AzureMachineLearningFileSystem

fs = AzureMachineLearningFileSystem('azureml://subscriptions/<SUB-ID>/resourcegroups/<RESOURCE-GROUP>/workspaces/<WORKSPACE-NAME>/datastores/workspaceworkingdirectory/paths/')

# 你可以将递归设置为 False 来上传文件
fs.upload(lpath='C:\\Users\\Downloads\\owid-covid-data.csv', rpath='data/fsspec', recursive=False, {'overwrite': 'MERGE_WITH_OVERWRITE'})

上述代码已执行,并将文件上传到默认数据存储。

输出:

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

对于第二个问题,如果你处理大量非结构化数据,Blob 存储 可能是更好的选择。如果你需要更高级别的访问控制并且处理较小量的结构化数据,文件共享 可能是更好的选择。

参考链接:
在交互式开发期间访问 Azure 云存储中的数据 - Azure 机器学习 | 微软学习

英文:

> NotImplementedError: I haven't information about this problem...

The NotImplementedError occurs here while using the AzureMachineLearningFileSystem class to upload a file is most likely the result of a version conflict between the azureml-sdk and fsspec packages. To check whether it fixes the problem, try updating both packages to the latest or relevant versions.

I tried with the same code to upload a file as you have selected the default directory(File share directory).

Code:

from azureml.fsspec import AzureMachineLearningFileSystem

fs = AzureMachineLearningFileSystem(&#39;azureml://subscriptions/&lt;SUB-ID&gt;/resourcegroups/&lt;RESOURCE-GROUP&gt;/workspaces/&lt;WORKSPACE-NAME&gt;/datastores/workspaceworkingdirectory/paths/&#39;)

# you can specify recursive as False to upload a file
fs.upload(lpath=&#39;C:\\Users\\Downloads\\owid-covid-data.csv&#39;, rpath=&#39;data/fsspec&#39;, recursive=False, **{&#39;overwrite&#39;: &#39;MERGE_WITH_OVERWRITE&#39;})

The above code executed and uploaded the file in the default data store.

Output:

update 函数中的 NotImplementedError 使用 AzureMachineLearningFileSystem 时发生。

For the second question,Blob storage can be a better option if you work with a lot of unstructured data.File sharing could be the better choice if you need higher levels of access control and are working with smaller volumes of structured data.

Reference:
Access data from Azure cloud storage during interactive development - Azure Machine Learning | Microsoft Learn

huangapple
  • 本文由 发表于 2023年7月3日 19:27:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/76604294.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定