最佳方法来构建一个需要大量参数的长期Python项目是什么?

huangapple go评论66阅读模式
英文:

What is the best way to structure a long-term python project that requires lots of arguments?

问题

我有一些具有不同功能的Python模块,我希望它们可以单独运行或作为更大数据管道的一部分运行。我已经组织了代码,以便使用以下大致布局来实现这一点:

dataProcessing.py
import...
def main():....
if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    args = parser.parse_args([])
    parser.add_argument("--arg1").....
dataPipeline.py
import dataProcessing
def main():
    dataProcessing(args)
if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    args = parser.parse_args([])
    parser.add_argument("--arg1").....

这种方式使我能够将模块排列在一个管道中并分别运行它们,这对项目是必要的。然而,这意味着参数需要在两个模块中定义,这点我可以接受。这对于命令行参数来说没问题,但当我稍后回到这些文件时会更加困难。是否有一种方法可以在一个文件中预先定义参数,以便我可以轻松地回到文件并进行编辑,而不是在命令行中进行编辑?我认为配置文件可能是合适的,但我对我的一般方法有一些疑虑,所以如果有关最佳实践的任何建议,将不胜感激。

英文:

I have a number of python modules with different functionality which I want to run on their own or as part of a larger data pipeline. I've organised the code to do this with the following rough layout:

dataProcessing.py
import...
def main():....
if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    args = parser.parse_args([])
    parser.add_argument("--arg1").....
dataPipeline.py
import dataProcessing
def main():
    dataProcessing(args)
if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    args = parser.parse_args([])
    parser.add_argument("--arg1").....

In this way I can arrange the modules in a pipeline and run them separately which is necessary for the project. However this does mean that the arguments need to be defined in both modules, which I can live with. This is okay for command line arguments, but harder when I come back to the files sometime later. Is there some way to have a file with the arguments already defined in so I it's easy to come back to the file and easier to edit rather than in the command line? I think a config file would be suitable but I am a bit doubtful about my general approach so any advice on best practice would be really appreciated.

答案1

得分: 1

是的,您可以只需定义一个新的.py文件,在其中定义您的参数。例如,创建一个名为params.py的文件,然后在此文件中定义一个变量,如下所示:

var = 'sample_string.txt'

在您的其他Python文件中,您可以像这样导入:

import params as p

然后您可以如下使用参数:

samplefilename_from_params_file = p.var

这样可以让您将所有参数集中在一个文件中。但是,还有其他方法来处理这个问题,例如使用一个真正的配置文件,可以是一个INI文件或一个yaml文件。

英文:

Yes, you can just define a new .py file, where you define your arguments/parameters.
For instance, create a file params.py
and inside this file you would define a variable like:

var = 'sample_string.txt'

In your other python files you would import, for instance like that:

import params as p

and then you can use the arguments as such:

samplefilename_from_params_file = p.var

This allows you to have all arguments condensed in one file.
However, there are other methods to deal with that and have a real configuration file. That could be a INI file or a yaml file.

huangapple
  • 本文由 发表于 2023年7月6日 20:13:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/76628711.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定