英文:
What is the best way to structure a long-term python project that requires lots of arguments?
问题
我有一些具有不同功能的Python模块,我希望它们可以单独运行或作为更大数据管道的一部分运行。我已经组织了代码,以便使用以下大致布局来实现这一点:
dataProcessing.py
import...
def main():....
if __name__ == "__main__":
parser = argparse.ArgumentParser()
args = parser.parse_args([])
parser.add_argument("--arg1").....
dataPipeline.py
import dataProcessing
def main():
dataProcessing(args)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
args = parser.parse_args([])
parser.add_argument("--arg1").....
这种方式使我能够将模块排列在一个管道中并分别运行它们,这对项目是必要的。然而,这意味着参数需要在两个模块中定义,这点我可以接受。这对于命令行参数来说没问题,但当我稍后回到这些文件时会更加困难。是否有一种方法可以在一个文件中预先定义参数,以便我可以轻松地回到文件并进行编辑,而不是在命令行中进行编辑?我认为配置文件可能是合适的,但我对我的一般方法有一些疑虑,所以如果有关最佳实践的任何建议,将不胜感激。
英文:
I have a number of python modules with different functionality which I want to run on their own or as part of a larger data pipeline. I've organised the code to do this with the following rough layout:
dataProcessing.py
import...
def main():....
if __name__ == "__main__":
parser = argparse.ArgumentParser()
args = parser.parse_args([])
parser.add_argument("--arg1").....
dataPipeline.py
import dataProcessing
def main():
dataProcessing(args)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
args = parser.parse_args([])
parser.add_argument("--arg1").....
In this way I can arrange the modules in a pipeline and run them separately which is necessary for the project. However this does mean that the arguments need to be defined in both modules, which I can live with. This is okay for command line arguments, but harder when I come back to the files sometime later. Is there some way to have a file with the arguments already defined in so I it's easy to come back to the file and easier to edit rather than in the command line? I think a config file would be suitable but I am a bit doubtful about my general approach so any advice on best practice would be really appreciated.
答案1
得分: 1
是的,您可以只需定义一个新的.py
文件,在其中定义您的参数。例如,创建一个名为params.py
的文件,然后在此文件中定义一个变量,如下所示:
var = 'sample_string.txt'
在您的其他Python文件中,您可以像这样导入:
import params as p
然后您可以如下使用参数:
samplefilename_from_params_file = p.var
这样可以让您将所有参数集中在一个文件中。但是,还有其他方法来处理这个问题,例如使用一个真正的配置文件,可以是一个INI文件或一个yaml文件。
英文:
Yes, you can just define a new .py
file, where you define your arguments/parameters.
For instance, create a file params.py
and inside this file you would define a variable like:
var = 'sample_string.txt'
In your other python files you would import
, for instance like that:
import params as p
and then you can use the arguments as such:
samplefilename_from_params_file = p.var
This allows you to have all arguments condensed in one file.
However, there are other methods to deal with that and have a real configuration file
. That could be a INI
file or a yaml
file.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论