英文:
PowerShell script to get user information takes hours to run--any way to do it faster?
问题
I'm here to help with your translation request. Here's the translated portion of your text:
这个 PowerShell 脚本用于从 Microsoft 365 获取大量用户信息并写入 CSV 文件,运行良好,但对于约 12,000 个用户来说需要很长时间(几个小时)。是否有更快的方法来完成这个任务?是否可以从 Python 中调用某个 API 来更快地完成这个任务?
# 从 AAD 获取所有用户
$all = get-azureaduser -All $true
# 清空我们将使用的输出文件
$path = 'c:\blahblahpath\M365users.csv'
$path -replace ' ', '` '
Clear-Content -Path $path
# 循环遍历所有用户,获取属性并写入 CSV
$all | foreach-object {
$user = $_
$mbx = ""
$fwd = ""
$mbx = get-mailbox $user.userprincipalname -ErrorAction SilentlyContinue
$fwd = $mbx.forwardingsmtpaddress
$extem = $mbx.externalemailaddress
$aliases = $mbx.emailaddresses
$Extension_Attributes = New-Object Psobject -Property $user.ExtensionProperty
# 合并用户对象和扩展属性的所需属性到单个对象
$u_properties = [pscustomobject] @{
"UserPrincipalName" = $user.UserPrincipalName
"GivenName" = $user.GivenName
"Surname" = $user.Surname
"M3-DisplayName" = $user.DisplayName
"JobTitle" = $user.JobTitle
"Department" = $user.Department
"CompanyName" = $user.CompanyName
"M3-Mail" = $user.mail
"M3-OtherMails" = $user.OtherMails[0]
"ForwardingSmtpAddress" = $fwd
"ExternalEmailAddress" = $extem
"Aliases" = $aliases
"DeliverToMailboxAndForward" = $keepinmbx
"TelephoneNumber" = $user.TelephoneNumber
"PhysicalDeliveryOfficeName" = $user.PhysicalDeliveryOfficeName
"PreferredLanguage" = $user.PreferredLanguage
"MailNickName" = $user.MailNickName
"StreetAddress" = $user.StreetAddress
"City" = $user.City
"State" = $user.State
"PostalCode" = $user.PostalCode
"Country" = $user.Country
"UsageLocation" = $user.UsageLocation
"Mobile" = $user.Mobile
"UserType" = $user.UserType
"AccountEnabled" = $user.AccountEnabled
"LastLogon" = $user.LastLogon
"BlockCredential" = $user.BlockCredential
"AssignedLicenses" = $user.AssignedLicenses
"Created" = $Extension_Attributes.createdDateTime
"MemberNumber" = $Extension_Attributes.EmployeeID
"TokenExp" = $user.RefreshTokensValidFromDateTime
"OLDMemNumExt" = $Extension_Attributes.extension_60a1274a0a9d4344bd172d81b06d0f50_MemberNumber
"Source" = $Extension_Attributes.extension_06f8dc5952b048a1879af4e793844173_Source
"M3-MemType" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_MemType
"M3-Status" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_Status
"EmailTeams" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailTeams
"M3-EmailCC" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailCC
"M3-EmailPri" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailPri
"Cert" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_Cert
"M3-ExpDate" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_ExpDate
"WGTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_WGTitle
"ChapTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_ChapTitle
"OtherTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_LdrTitle
"Othercodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_Othercodes
"ChapCodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_ChapCodes
"WGcodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_WGcodes
"Allcodes" = $extension_Attributes.extension_a14f14664c8e457a856d24680b31c9d4_AllCodes
"Cmems" = $Extension_Attributes.extension_4adef2735ffb4a9dafc9a85188ca512d_Cmems
"M3-isnet" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_IsNet
"SipProxyAddress" = $user.SipProxyAddress
}
# 导出到 CSV
$u_properties | Export-csv -Path $path -NoTypeInformation -Force -Append
}
Regarding your question about the script's performance, the slow part seems to be within the loop where you retrieve mailbox information with get-mailbox
. You might explore optimizing this part for better performance, but it's a bit complex and depends on the specifics of your environment and requirements. You could potentially consider using Microsoft Graph API from Python to fetch user information in a more efficient way, but that would involve rewriting the script. If you decide to go that route, you can refer to Microsoft's documentation on using the Graph API for such tasks.
英文:
This PowerShell script to get a lot of user information from Microsoft 365 and write to csv works fine, but is taking a LONG time to run (like hours) for about 12k users. Is there any way to do this in a faster way? Is there an API I can call from Python that would do this faster?
# Get all users from AAD
$all = get-azureaduser -All $true
# Empty the output file we will use
$path = 'c:\blahblahpath\M365users.csv'
$path -replace ' ', '` '
Clear-Content -Path $path
# Loop through all users, get attributes and write to csv
$all | foreach-object {
$user = $_
$mbx = ""
$fwd = ""
$mbx = get-mailbox $user.userprincipalname -ErrorAction SilentlyContinue
$fwd = $mbx.forwardingsmtpaddress
$extem = $mbx.externalemailaddress
$aliases = $mbx.emailaddresses
# Expanding only the Extension Attributes related to the user and converting the Dictionary to Custom Object
# so that keys can be accessed through the dot (.) operator
$Extension_Attributes = New-Object Psobject -Property $user.ExtensionProperty
# Combining the required attributes from the user object and extension_attributes to A single object
$u_properties = [pscustomobject] @{
"UserPrincipalName" = $user.UserPrincipalName
"GivenName" = $user.GivenName
"Surname" = $user.Surname
"M3-DisplayName" = $user.DisplayName
"JobTitle" = $user.JobTitle
"Department" = $user.Department
"CompanyName" = $user.CompanyName
"M3-Mail" = $user.mail
"M3-OtherMails" = $user.OtherMails[0]
"ForwardingSmtpAddress" = $fwd
"ExternalEmailAddress" = $extem
"Aliases" = $aliases
"DeliverToMailboxAndForward" = $keepinmbx
"TelephoneNumber" = $user.TelephoneNumber
"PhysicalDeliveryOfficeName" = $user.PhysicalDeliveryOfficeName
"PreferredLanguage" = $user.PreferredLanguage
"MailNickName" = $user.MailNickName
"StreetAddress" = $user.StreetAddress
"City" = $user.City
"State" = $user.State
"PostalCode" = $user.PostalCode
"Country" = $user.Country
"UsageLocation" = $user.UsageLocation
"Mobile" = $user.Mobile
"UserType" = $user.UserType
"AccountEnabled" = $user.AccountEnabled
"LastLogon" = $user.LastLogon
"BlockCredential" = $user.BlockCredential
"AssignedLicenses" = $user.AssignedLicenses
"Created" = $Extension_Attributes.createdDateTime
"MemberNumber" = $Extension_Attributes.EmployeeID
"TokenExp" = $user.RefreshTokensValidFromDateTime
"OLDMemNumExt" = $Extension_Attributes.extension_60a1274a0a9d4344bd172d81b06d0f50_MemberNumber
"Source" = $Extension_Attributes.extension_06f8dc5952b048a1879af4e793844173_Source
"M3-MemType" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_MemType
"M3-Status" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_Status
"EmailTeams" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailTeams
"M3-EmailCC" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailCC
"M3-EmailPri" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_EmailPri
"Cert" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_Cert
"M3-ExpDate" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_ExpDate
"WGTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_WGTitle
"ChapTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_ChapTitle
"OtherTitle" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_LdrTitle
"Othercodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_Othercodes
"ChapCodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_ChapCodes
"WGcodes" = $extension_Attributes.extension_11b052b5eb8f4c77973a06876d25a6a4_WGcodes
"Allcodes" = $extension_Attributes. extension_a14f14664c8e457a856d24680b31c9d4_AllCodes
"Cmems" = $Extension_Attributes.extension_4adef2735ffb4a9dafc9a85188ca512d_Cmems
"M3-isnet" = $Extension_Attributes.extension_01816427922e40c9b55cf4364bc171ce_IsNet
"SipProxyAddress" = $user.SipProxyAddress
}
# check
#write-host "got $u_properties"
$u_properties | Export-csv -Path $path -NoTypeInformation -Force -Append
}#
I timed the $all = get-azureaduser -All $true
at about 20 seconds, so obviously the time is in the loop--maybe the 'get-mailbox...' is slow?
I'm fairly novice at PowerShell so I can't think of a lot of other ways to try doing this.
答案1
得分: 1
以下是翻译好的部分:
"I believe with simply putting Export-Csv
as the last statement of your pipeline your script time will improve greatly. Appending to a Csv means that in each loop iteration Export-Csv
has to open the FileStream
, seek until the EOF, append the new line (a single object) and close the stream. This is a really expensive thing to do 12k times. Instead, if you put it as the last statement, no -Append
is needed and the FileStream
is kept opened until no more output is coming from your ForEach-Object
."
"将Export-Csv
简单地放在管道的最后一条语句,您的脚本运行时间将大大提高。追加到 Csv 意味着在每次循环迭代中,Export-Csv
必须打开 FileStream
,查找直到 EOF,追加新行(一个对象),然后关闭流。这是一个非常昂贵的操作,需要执行 12,000 次。相反,如果将其放在最后一条语句,就不需要使用 -Append
,并且FileStream
将一直保持打开状态,直到来自ForEach-Object
的输出不再存在。"
"To put it into perspective how bad is -Append
inside a loop, you can run the following test:"
"为了更清楚地展示在循环中使用-Append
有多糟糕,您可以运行以下测试:"
然后是代码部分,我不会翻译代码。
英文:
I believe with simply putting Export-Csv
as the last statement of your pipeline your script time will improve greatly. Appending to a Csv means that in each loop iteration Export-Csv
has to open the FileStream
, seek until the EOF, append the new line (a single object) and close the stream. This is a really expensive thing to do 12k times. Instead, if you put it as the last statement, no -Append
is needed and the FileStream
is kept opened until no more output is coming from your ForEach-Object
.
Get-AzureADUser -All $true | ForEach-Object {
# same logic here
} | Export-Csv -Path $path -NoTypeInformation # here is the last statement, no append needed.
To put it into perspective how bad is -Append
inside a loop, you can run the following test:
$test = @{
'With Append' = {
$tmp = New-TemporaryFile
0..12000 | ForEach-Object {
[pscustomobject]@{
foo = 'bar'
} | Export-Csv $tmp.FullName -Append
}
$tmp | Remove-Item
}
'Pipeline Processing' = {
$tmp = New-TemporaryFile
0..12000 | ForEach-Object {
[pscustomobject]@{
foo = 'bar'
}
} | Export-Csv $tmp.FullName
$tmp | Remove-Item
}
}
$test.GetEnumerator() | ForEach-Object {
[pscustomobject]@{
Test = $_.Key
Time = (Measure-Command { & $_.Value }).TotalMilliseconds
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论