使用Semaphore和Register-ObjectEvent来限制进程数量,问题出在哪里?

huangapple go评论76阅读模式
英文:

Use Semaphore and Register-ObjectEvent to throttle the number of processes, where is the problem?

问题

以下是您的代码。基本上使用了Semaphore和Register-ObjectEvent来限制ffmpeg进程的数量。我有很多视频文件,但是在任何时候,只能处理其中两个。

$working_dir = "C:\temp\video\test"
$input_files = @(Get-ChildItem -Path $working_dir -Include @("*.webm", "*.mkv") -Recurse -File)

$throttleLimit = 2
$semaphore = [System.Threading.Semaphore]::new($throttleLimit, $throttleLimit)
$jobs = @()

foreach ($v in $input_files) {
    $new_name = [regex]::replace($v.fullname, '\.[^.]+$', '') + ".mp4"
    $ArgumentList = "-i `"$($v.fullname)`" -metadata comment= -metadata title= -filter_complex `"`drawtext=fontsize=10:fontfile='I:\temp\sarasa-term-sc-bold.ttf':text=''':x=228:y=840:fontcolor=000000`"` -c:a copy -y `"$($new_name)`""

    $semaphore.WaitOne()

    $process = Start-Process -FilePath "ffmpeg.exe" -ArgumentList $ArgumentList -PassThru -NoNewWindow

    $job = Register-ObjectEvent -InputObject $process -EventName "Exited" -Action {
        $semaphore.Release()
        Unregister-Event $eventSubscriber.SourceIdentifier
        Remove-Job $eventSubscriber.Action
    }

    $jobs += $job
}

$jobs | Wait-Job | Receive-Job

我注意到的问题是,只有两个文件被ffmpeg处理了。当前两个视频文件被处理时,我贴上了上述代码片段的PowerShell控制台如下所示。

使用Semaphore和Register-ObjectEvent来限制进程数量,问题出在哪里?

看起来ffmpeg没有退出。但是当我从另一个PowerShell会话中检查时,没有找到ffmpeg进程。

PS C:\WINDOWS\system32> ps ffmpeg
ps : 找不到名称为"ffmpeg"的进程请验证进程名称然后再次调用cmdlet
在行:1 字符:1
+ ps ffmpeg
+ ~~~~~~~
    + CategoryInfo          : ObjectNotFound: (ffmpeg:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

PS C:\WINDOWS\system32> ps *ffmpeg*
PS C:\WINDOWS\system32>

问题出在哪里?我的代码逻辑是否正确?

英文:

Below is my code. It basically uses a Semaphore and and Register-ObjectEvent to throttle the number of ffmpeg processes. I have a lot of video files, but at anytime, only two of them should being processed.

$working_dir = "C:\temp\video\test"
$input_files = @(Get-ChildItem -Path $working_dir -Include @("*.webm", "*.mkv") -Recurse -File)

$throttleLimit = 2
$semaphore = [System.Threading.Semaphore]::new($throttleLimit, $throttleLimit)
$jobs = @()

foreach ($v in $input_files) {
    $new_name = [regex]::replace($v.fullname, '\.[^\.]+$', '') + ".mp4"
    $ArgumentList = "-i `"$($v.fullname)`" -metadata comment= -metadata title= -filter_complex `"drawtext=fontsize=10:fontfile='I\:\\temp\\sarasa-term-sc-bold.ttf':text='':x=228:y=840:fontcolor=000000`" -c:a copy -y `"$($new_name)`""

    $semaphore.WaitOne()

    $process = Start-Process -FilePath "ffmpeg.exe" -ArgumentList $ArgumentList -PassThru -NoNewWindow

    $job = Register-ObjectEvent -InputObject $process -EventName "Exited" -Action {
        $semaphore.Release()
        Unregister-Event $eventSubscriber.SourceIdentifier
        Remove-Job $eventSubscriber.Action
    }

    $jobs += $job
}

$jobs | Wait-Job | Receive-Job

The problem I noticed is that, only two files got processed by ffmpeg. The PowerShell console that I pasted the upper code snippet looks like this when the first two video files got processed.

使用Semaphore和Register-ObjectEvent来限制进程数量,问题出在哪里?

It looks like ffmpeg didn't exit. But when I check from another PowerShell session, no ffmpeg process was found.

PS C:\WINDOWS\system32> ps ffmpeg
ps : Cannot find a process with the name "ffmpeg". Verify the process name and call the cmdlet again.
At line:1 char:1
+ ps ffmpeg
+ ~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (ffmpeg:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

PS C:\WINDOWS\system32> ps *ffmpeg*
PS C:\WINDOWS\system32>

Where is the problem? Is the logic in my code correct?

答案1

得分: 1

您的代码存在两个问题:

  1. Action块不知道$semaphore是什么,您需要通过-MessageData传递此实例的引用,并使用$event.MessageData进行回调。

  2. $semaphore.WaitOne()会无限期地锁定线程,WaitOne()不会允许PowerShell检查中断。基本上,您需要将此调用更改为具有超时的循环。

这里是一个简单的工作示例,限制同时打开的notepad进程数量。我还将您的Semaphore更改为更好的版本

$lock = [System.Threading.SemaphoreSlim]::new(2, 2)
$jobs = foreach($i in 0..10) {
    while(-not $lock.Wait(200)) { }
    $proc = Start-Process notepad -PassThru

    $registerObjectEventSplat = @{
        InputObject = $proc
        EventName   = 'Exited'
        MessageData = $lock
        Action      = {
            $event.MessageData[0].Release()
            Unregister-Event $eventSubscriber.SourceIdentifier
            Remove-Job $eventSubscriber.Action
        }
    }
    Register-ObjectEvent @registerObjectEventSplat
}

一个更简单的方法是使用PowerShell 5.1中提供的ThreadJob模块,在较新版本的PowerShell 7+中已预安装。Start-ThreadJob cmdlet具有内置的节流机制,大大简化了上述过程。供参考的比较示例:

$jobs = foreach($i in 0..10) {
    Start-ThreadJob {
        Start-Process notepad -Wait
    } -ThrottleLimit 2
}
$jobs | Receive-Job -AutoRemoveJob -Wait
英文:

Your code has 2 problems:

  1. The Action block does not know what $semaphore is, you need to pass the reference of this instance with -MessageData and call it back using $event.MessageData.

  2. $semaphore.WaitOne() will lock the thread indefinitely, WaitOne() won't allow PowerShell to check for interrupts. Basically you need to change this call for a loop with a timeout.

Here is a simple working example, throttling the number of notepad processes that can be opened at the same time. I have also changed your Semaphore for a better version of it.

$lock = [System.Threading.SemaphoreSlim]::new(2, 2)
$jobs = foreach($i in 0..10) {
    while(-not $lock.Wait(200)) { }
    $proc = Start-Process notepad -PassThru

    $registerObjectEventSplat = @{
        InputObject = $proc
        EventName   = 'Exited'
        MessageData = $lock
        Action      = {
            $event.MessageData[0].Release()
            Unregister-Event $eventSubscriber.SourceIdentifier
            Remove-Job $eventSubscriber.Action
        }
    }
    Register-ObjectEvent @registerObjectEventSplat
}

A much easier approach would be using the ThreadJob Module available through the Gallery for PowerShell 5.1 and pre-installed in newer versions of PowerShell 7+. The Start-ThreadJob cmdlet has a built-in throttling mechanism that simplifies the above process a lot. For comparison:

$jobs = foreach($i in 0..10) {
    Start-ThreadJob {
        Start-Process notepad -Wait
    } -ThrottleLimit 2
}
$jobs | Receive-Job -AutoRemoveJob -Wait

huangapple
  • 本文由 发表于 2023年6月2日 10:30:47
  • 转载请务必保留本文链接:https://go.coder-hub.com/76386792.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定