混合使用Python和Go语言

huangapple go评论99阅读模式
英文:

Mixing Python and Go

问题

我一直在使用Python编写一个库,我想要进行一些性能改进。

是否可以在Python中编写一些代码,在Go中编写一些代码,并在它们之间传递数据?
如果可以的话,是否有任何关于如何做到这一点的示例?

就像这样:

# Python
def python_foo():
    data = {'foo': 'val', 'bar': [1, 2, 3]}
    go_process(json.dumps(data))


def python_got_data_from_go(data):
    # 处理来自Go的数据


# Go
func go_process(json string) {
    // 进行一些处理
    python_got_data_from_go(someData)
}
英文:

I have been working on a library in Python and I would like to do some performance improvement.

Is it possible to write some code in Python and some code in Go, and pass data between them?
And if it's possible, are there any examples on how to do this?

Like such:

# Python
def python_foo():
    data = {'foo': 'val', 'bar': [1, 2, 3]}
    go_process(json.dumps(data))


def python_got_data_from_go(data):
    # deal with data from Go


# Go
func go_process(json string) {
    // do some processing
    python_got_data_from_go(someData)
}

答案1

得分: 10

你需要一个“胶水”来连接它们,例如使用C编程语言或通过网络进行通信。
如果你将https://docs.python.org/2/extending/extending.html与http://golang.org/cmd/cgo/混合使用,并且具备良好的C编程技巧,那么这将是一个非常痛苦的解决方案。

你可以在Python中创建服务器,参考http://pymotw.com/2/socket/tcp.html,在Go中创建服务器,参考https://coderwall.com/p/wohavg,并在它们之间进行通信。

编辑:
请参考https://stackoverflow.com/questions/12443203/writing-a-python-extension-in-go-golang?lq=1和https://stackoverflow.com/questions/19397986/calling-python-function-from-go-and-getting-the-function-return-value?lq=1。

英文:

You need a glue between them, for example C programming language or communication over network.
Most painful solution if you mix https://docs.python.org/2/extending/extending.html with http://golang.org/cmd/cgo/ and good programming skills in C.

You might create server in python http://pymotw.com/2/socket/tcp.html and in go https://coderwall.com/p/wohavg and communicate between them.

Edit:
see https://stackoverflow.com/questions/12443203/writing-a-python-extension-in-go-golang?lq=1 and https://stackoverflow.com/questions/19397986/calling-python-function-from-go-and-getting-the-function-return-value?lq=1

答案2

得分: 3

混合使用多种语言的最简单方法是通过套接字进行通信,例如TCP/IP或Unix域套接字,或通过高级协议如HTTP或XML-RPC。然而,这种方法会带来很高的开销,因为需要进行请求处理和JSON/XML的序列化/反序列化,如果需要进行大量调用,这个开销可能会很大。如果每个请求的工作量很大,通常使用套接字进行通信是最好的选择。

如果你不想承担套接字的开销(比如每秒钟在Python和Go之间进行数千次请求),还有其他可能开销更低的解决方案。你可以尝试在支持的操作系统中使用共享内存。共享内存通常具有更低廉的数据访问成本,但可能需要将共享内存结构转换为Python数据类型,这可能会带来一些开销。此外,请注意你可能需要自己管理锁。

除非你只进行非常少量的调用,并且这些调用不需要在调用之间共享状态,否则我不建议使用标准的stdin/stdout进行通信。

最后一种选择是编写Python扩展;我不建议对于不熟悉的人来说这是一个简单的方法。

英文:

The easiest way to mix languages is to to communicate over a socket, such as TCP/IP or Unix domain socket or through a high level protocol like HTTP or XML-RPC. This comes with very high overhead though due to the request processing and serialization/deserialization to/from JSON/XML, which can be significant if you have to make lots of calls. Communicating over socket is usually best if the amount of workload per request is high.

If you are not willing to pay for the overhead of socket (say if you make thousands of requests back and forth between python and go per seconds), there are other solutions that may have lower overhead. You may be able to use shared memory in OSes that have them. Shared memory usually incurs much cheaper cost to access data, but it may incur boxing/unboxing cost from the shared memory structure to Python datatypes. Also, note that you may have to manage the locks yourself with this.

Unless you only make very small number of calls and they do not need to share state between calls I would not recommend communicating using the standard stdin/stdout.

The last alternative is to write Python Extension; I would not recommend this for the faint of heart.

答案3

得分: 2

最简单的解决方案是从Python中生成一个Go进程,并通过Go的进程标准流(os.Stdinos.Stdout)进行通信。你需要发明一个双方都同意的协议(看起来你已经选择了JSON),并且在从两端流式传输逻辑上原子请求后不要忘记刷新流。

这样,你的解决方案(几乎)是跨平台的,并且非常容易设置。

英文:

The simplest solution is to spawn a Go process from Python and make them communicate via the Go's process standard streams (os.Stdin and os.Stdout). You have to invent a protocol both sides agree on (looks like you've settled on JSON already) and not to forget to flush the stream after streaming a logically atomic request, from both ends.

This way your solution is (almost) cross-platform and very easy to set up.

huangapple
  • 本文由 发表于 2014年4月27日 00:51:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/23313915.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定