英文:
NotImplementedError in update with AzureMachineLearningFileSystem
问题
I have a problem with upload on azure
https://learn.microsoft.com/pl-pl/azure/machine-learning/how-to-access-data-interactive?view=azureml-api-2&tabs=adls
I use sample
from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<workspace_name>/datastores/<datastorename>/paths/')
# you can specify recursive as False to upload a file
fs.upload(lpath='data/upload_files/crime-spring.csv', rpath='data/fsspec', recursive=False, **{'overwrite': 'MERGE_WITH_OVERWRITE'})
# you need to specify recursive as True to upload a folder
fs.upload(lpath='data/upload_folder/', rpath='data/fsspec_folder', recursive=True, **{'overwrite': 'MERGE_WITH_OVERWRITE'})
This is offical solution.
When i try
from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/'+auth_subscription_id+'/resourcegroups/'+auth_resource_group+'/workspaces/'+auth_workspace_name+'/datastores/azure_churn_lab_datastores/paths/')
# you can specify recursive as False to upload a file
fs.upload(lpath='/' + path + seq_name + '.csv',
rpath='/' + path_target)
I have error with NotImplementedError:
I havent information about this problem... I tried upgrade package azure sdk.
I want ask you - this is problem with implementation or azure?
I know about different way to upload with folder.
Second question
If you prepare data in python sdk2 and default path is
You upload to blob containers.
or leave in this path (File Share)? And why? Because I saw - few people save and upload
I use Visual Studio Code - connect to
and when I save a file
df.to_csv('/test.csv')
default path is File Share code
英文:
I have a problem with upload on azure
https://learn.microsoft.com/pl-pl/azure/machine-learning/how-to-access-data-interactive?view=azureml-api-2&tabs=adls
I use sample
from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/<subid>/resourcegroups/<rgname>/workspaces/<workspace_name>/datastores/<datastorename>/paths/')
# you can specify recursive as False to upload a file
fs.upload(lpath='data/upload_files/crime-spring.csv', rpath='data/fsspec', recursive=False, **{'overwrite': 'MERGE_WITH_OVERWRITE'})
# you need to specify recursive as True to upload a folder
fs.upload(lpath='data/upload_folder/', rpath='data/fsspec_folder', recursive=True, **{'overwrite': 'MERGE_WITH_OVERWRITE'})
This is offical solution.
When i try
from azureml.fsspec import AzureMachineLearningFileSystem
# instantiate file system using following URI
fs = AzureMachineLearningFileSystem('azureml://subscriptions/'+auth_subscription_id+'/resourcegroups/'+auth_resource_group+'/workspaces/'+auth_workspace_name+'/datastores/azure_churn_lab_datastores/paths/')
# you can specify recursive as False to upload a file
fs.upload(lpath='/' + path + seq_name + '.csv',
rpath='/' + path_target)
I have error with NotImplementedError:
I havent information about this problem... I tried upgrade package azure sdk.
I want ask you - this is problem with implementation or azure?
I know about different way to upload with folder.
Second question
If you prepare data in python sdk2 and default path is
You upload to blob containers.
or leave in this path (File Share)? And why? Because I saw - few people save and upload
I use Visual Studio Code - connect to
and when I save a file
df.to_csv('/test.csv')
default path is File Share code
答案1
得分: 1
NotImplementedError: I haven't information about this problem...
在使用
AzureMachineLearningFileSystem
类上传文件时出现NotImplementedError
错误,这很可能是azureml-sdk
和 **fsspec
包之间版本冲突的结果。为了检查是否解决了问题,请尝试将这两个包都更新到最新或相关版本。我尝试使用相同的代码上传文件,因为你选择了默认目录(文件共享目录)。
代码:
from azureml.fsspec import AzureMachineLearningFileSystem fs = AzureMachineLearningFileSystem('azureml://subscriptions/<SUB-ID>/resourcegroups/<RESOURCE-GROUP>/workspaces/<WORKSPACE-NAME>/datastores/workspaceworkingdirectory/paths/') # 你可以将递归设置为 False 来上传文件 fs.upload(lpath='C:\\Users\\Downloads\\owid-covid-data.csv', rpath='data/fsspec', recursive=False, {'overwrite': 'MERGE_WITH_OVERWRITE'})
上述代码已执行,并将文件上传到默认数据存储。
输出:
对于第二个问题,如果你处理大量非结构化数据,
Blob 存储
可能是更好的选择。如果你需要更高级别的访问控制并且处理较小量的结构化数据,文件共享
可能是更好的选择。
英文:
> NotImplementedError: I haven't information about this problem...
The NotImplementedError
occurs here while using the AzureMachineLearningFileSystem
class to upload a file is most likely the result of a version conflict between the azureml-sdk
and fsspec packages
. To check whether it fixes the problem, try updating both packages to the latest or relevant versions.
I tried with the same code to upload a file as you have selected the default directory(File share directory).
Code:
from azureml.fsspec import AzureMachineLearningFileSystem
fs = AzureMachineLearningFileSystem('azureml://subscriptions/<SUB-ID>/resourcegroups/<RESOURCE-GROUP>/workspaces/<WORKSPACE-NAME>/datastores/workspaceworkingdirectory/paths/')
# you can specify recursive as False to upload a file
fs.upload(lpath='C:\\Users\\Downloads\\owid-covid-data.csv', rpath='data/fsspec', recursive=False, **{'overwrite': 'MERGE_WITH_OVERWRITE'})
The above code executed and uploaded the file in the default data store.
Output:
For the second question,Blob storage
can be a better option if you work with a lot of unstructured data.File sharing
could be the better choice if you need higher levels of access control and are working with smaller volumes of structured data.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论