英文:
How to get print statements to work while running multiprocessing in juypter notebook
问题
在Jupyter Notebook中运行Python多进程时，我将我的函数放在另一个.py文件中，
在那个函数中，我使用了打印语句。
将函数放在.py文件中允许我在Jupyter中使用多进程，但它不会在Jupyter中打印出来。
有没有办法我们可以实现这一点？
def func(x):
print(f'Running for {x}')
return x * x
假设这是函数...我希望我的打印函数能够工作。
英文:
While Running python Multiprocessing in Jupyter Notebook, I put my functions in another .py file,
In that function i use print statements.
keeping function in .py file allows me to use multiprocessing in Jupyter, but it does not print the in the jupyter.
Is there any way we can achieve this?
def func(x):
print(f'Running for {x}')
return x * x
suppose this is the function.... I want my print function to work.
答案1
得分: 1
当您启动Jupyter Notebook时，会打开两个窗口。一个在您的浏览器中，您可以在其中创建单元格，另一个是控制台窗口，Jupyter Notebook会在其中输出消息。当子进程执行print时，其输出会显示在控制台窗口中。
如果您希望输出显示在浏览器窗口中，一种方法是让子进程将其打印输出放到multiprocessing.Queue实例中，而不是直接打印它。您的单元格之前启动的线程会获取这些消息并将其打印出来。例如：
文件 workers.py
def initialize_pool(q):
global print_queue
print_queue = q
def func(x):
print_queue.put(f'Running for {x}')
return x * x
Jupyter Notebook 单元格
class Sentinel:
pass
def main():
from threading import Thread
from multiprocessing import Pool, Queue
from workers import initialize_pool, func
print_queue = Queue()
def printer():
while True:
msg = print_queue.get()
if isinstance(msg, Sentinel):
break
print(msg, flush=True)
t = Thread(target=printer)
t.start()
with Pool(initializer=initialize_pool, initargs=(print_queue,)) as pool:
results = pool.map(func, [1,2,3,4,5])
print_queue.put(Sentinel()) # 放置一个标志
t.join() # 等待所有打印完成
print(results, flush=True)
if __name__ == '__main__':
main()
在Jupyter Notebook的窗口中的输出：
Running for 1
Running for 3
Running for 4
Running for 5
Running for 2
[1, 4, 9, 16, 25]
英文:
When you start Jupyter Notebook there are two windows that are opened. One is in your browser where you create your cells and the other is a console window where Jupyter Notebook outputs its messages. When a child process does a print, its output goes to the console window.
If you must have the output go to the browser's window, then one method is to have the child process putting its print output to a multiprocessing.Queue instance instead of printing it directly. A thread previously started by your cell gets these messages and prints them. For example:
File workers.py
def initialize_pool(q):
global print_queue
print_queue = q
def func(x):
print_queue.put(f'Running for {x}')
return x * x
Jupyter Notebook Cell
class Sentinel:
pass
def main():
from threading import Thread
from multiprocessing import Pool, Queue
from workers import initialize_pool, func
print_queue = Queue()
def printer():
while True:
msg = print_queue.get()
if isinstance(msg, Sentinel):
break
print(msg, flush=True)
t = Thread(target=printer)
t.start()
with Pool(initializer=initialize_pool, initargs=(print_queue,)) as pool:
results = pool.map(func, [1,2,3,4,5])
print_queue.put(Sentinel()) # Put a sentinel
t.join() # Wait for all printing to complete
print(results, flush=True)
if __name__ == '__main__':
main()
Output in Jupyter Notebook's Window:
Running for 1
Running for 3
Running for 4
Running for 5
Running for 2
[1, 4, 9, 16, 25]
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论