Update Catalog scraping throws "You cannot call a method on a null-valued expression" and costs ~10 minutes per device

オープン
#376 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
50/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
静か
技術スタック
powershell
領域
devops

調査の方向性

Start with the MsUpCatResponse constructor in Classes/MsUpCatResponse.Class.ps1, where ctl00_catalogBody_updateMatches is read without a null check, then the empty-page guard in Private/MSCatalog/Invoke-CatalogRequest.ps1 around lines 28-31. Run Save-MsUpCatDriver or Get-MsUpCat against a search that returns no results table. Done means the lookup returns an empty result without the null-method warning, the request has a timeout, and any warning includes the URI and exception type.

索引モデルが issue の本文から書いたものです。

説明

Module version: OSD 26.7.6.1 (PSGallery, published 2026-07-06)
Affects: Invoke-OSDCloud -> Save-MsUpCatDriver -> Get-MsUpCat -> Invoke-CatalogRequest -> [MsUpCatResponse]
Related: #289 ("Unable to get driver for hardware component") appears to be the same code path, undiagnosed.

Symptom

During the "Microsoft Update Catalog Drivers" phase of an OSDCloud deployment, every device produces
six identical, context-free warnings and then a "No Results" line, taking almost exactly 10 minutes
per device:

[12:27:07 PM] Microsoft Update Catalog Drivers
Drivers for PNPClass DiskDrive will be downloaded from Microsoft Update Catalog to C:\Drivers\MsUpCatDrivers
Drivers for PNPClass Net will be downloaded from Microsoft Update Catalog to C:\Drivers\MsUpCatDrivers
WARNING: You cannot call a method on a null-valued expression.
WARNING: You cannot call a method on a null-valued expression.
WARNING: You cannot call a method on a null-valued expression.
WARNING: You cannot call a method on a null-valued expression.
WARNING: You cannot call a method on a null-valued expression.
WARNING: You cannot call a method on a null-valued expression.
[12:37:09 PM] [Save-MsUpCatDriver] No Results: Intel(R) Wi-Fi 6E AX211 160MHz VEN_8086&DEV_7E40
WARNING: You cannot call a method on a null-valued expression.
... x6 ...
[12:47:09 PM] [Save-MsUpCatDriver] No Results: Intel(R) Ethernet Connection (18) I219-LM VEN_8086&DEV_550A

Six warnings per device corresponds exactly to the six fallback searches in Save-MsUpCatDriver
(22H2, 21H2, Vibranium, 1903, 1809, bare). The warning names no function, no URL and no
hardware ID, which makes this very hard to attribute.

Upstream trigger: the catalog no longer returns the element being parsed

Reproduced directly against catalog.update.microsoft.com from a normal Windows 11 host, outside
WinPE, with Invoke-WebRequest -UseBasicParsing:

Query HTTP Response time ctl00_catalogBody_updateMatches ctl00_catalogBody_noResultText
22H2+Net+PCI\VEN_8086&DEV_550A 200 100.6s ABSENT ABSENT
KB5034123 (a KB that definitely exists) 200 100.5s ABSENT ABSENT
PCI\VEN_8086&DEV_550A 200 154.5s ABSENT ABSENT

A known-good KB search returns no results table either, so this is not a "no matches for this
hardware ID" condition. The only ctl00_catalogBody_* element ids remaining on the page are:

ctl00_catalogBody_ResultsHeaderTable
ctl00_catalogBody_searchString
ctl00_catalogBody_textDetailsPopupBlocked

__EVENTARGUMENT, __EVENTVALIDATION and __VIEWSTATE are still present. Page <title> is still
Microsoft Update Catalog, there is no error page and no 8DDD* error code. Responses consistently
take 100 to 155 seconds.

So the search results table id that the module depends on is gone site-wide, and the "no results"
element used to detect an empty search is gone as well.

Why the module turns that into a 10-minute hang and a cryptic message

Three separate issues compound it.

1. Classes/MsUpCatResponse.Class.ps1 has no null guards (lines 10-15).

MsUpCatResponse($HtmlDoc) {
    $Table = $HtmlDoc.GetElementbyId("ctl00_catalogBody_updateMatches")
    $this.Rows = $Table.SelectNodes("tr") | Where-Object { $_.Id -ne "headerRow" }   # <-- throws when $Table is $null
    $this.EventArgument = $HtmlDoc.GetElementbyId("__EVENTARGUMENT").Attributes["value"].Value
    $this.EventValidation = $HtmlDoc.GetElementbyId("__EVENTVALIDATION").Attributes["value"].Value
    $this.ViewState = $HtmlDoc.GetElementbyId("__VIEWSTATE").Attributes["value"].Value
    $this.ViewStateGenerator = $HtmlDoc.GetElementbyId("__VIEWSTATEGENERATOR").Attributes["value"].Value
    $NextPageNode = $HtmlDoc.GetElementbyId("ctl00_catalogBody_nextPageLink")
    $this.NextPage = if ($null -ne $NextPageNode) { $NextPageNode.InnerText.Trim() } else { $null }
}

NextPage is the only member that is null-checked. This is unchanged on master as of this report.

2. Private/MSCatalog/Invoke-CatalogRequest.ps1 cannot detect this page state, and discards all error context.

The guard at lines 28-31 only recognises two failure shapes:

$NoResults = $HtmlDoc.GetElementbyId("ctl00_catalogBody_noResultText")
$ErrorText = $HtmlDoc.GetElementbyId("errorPageDisplayedError")
if ($null -eq $NoResults -and $null -eq $ErrorText) {
    return [MsUpCatResponse]::new($HtmlDoc)      # <-- taken, because neither element exists any more
}

Both are absent now, so the page is treated as a valid result set and the constructor throws. The
handler at lines 46-48 then erases every useful detail:

catch {
    Write-Warning "$_"
}

There is also no -TimeoutSec on the Invoke-WebRequest at line 25, so each attempt waits out the
catalog's full 100s+ response.

3. Save-MsUpCatDriver retries six times per device with no failure memory.

Public/Functions/MicrosoftCatalog.ps1 lines 705-721 issue six sequential Get-MsUpCat calls per
device. There is no circuit breaker, so a site-wide outage is re-probed for every hardware ID in the
machine. At ~100s per request that is ~10 minutes per device. Public/OSDCloud.ps1 lines 1776-1786
runs this for DiskDrive, Net and SCSIAdapter, so a typical laptop pays this several times over.

Additionally, $Rows = $Rows.Where({...}) at MicrosoftCatalog.ps1 lines 252 and 369 operates on
$Rows = $Res.Rows (line 233) without checking whether $Res is $null.

Suggested fixes

  1. Null-guard the MsUpCatResponse constructor. If ctl00_catalogBody_updateMatches is absent,
    leave Rows empty rather than throwing. Same for the __VIEWSTATE family.
  2. In Invoke-CatalogRequest, treat "no results table" as an empty result rather than a valid
    response, add a -TimeoutSec, and include the URI and exception type in the warning instead of
    bare "$_".
  3. Add a per-run circuit breaker so that once the catalog is determined to be unreachable or
    unparseable, the remaining lookups short-circuit instead of re-probing per device.

Workaround (verified)

Replacing Invoke-CatalogRequest in the module's own scope with a version that adds a 20s timeout,
detects the missing results table, and latches a "broken" flag reduced the cost from ~10 minutes per
device to 20 seconds once for the entire deployment, with zero warnings. Measured: first call 20.1s,
second call 0s, five subsequent calls 0.03s total.

Environment

  • OSD 26.7.6.1
  • Deployment host: WinPE (Windows PowerShell 5.1)
  • Reproduced independently on Windows 11 with PowerShell 7.6.4, so this is not WinPE specific
  • Get-MsUpCat also affected directly, not only via Save-MsUpCatDriver
主要言語
PowerShell
スター
251
フォーク
102
平均マージ
2分
マージ済み PR(30日)
4

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

OSDeploy/OSD のほかの issue

OSDeploy/OSD の issue をすべて見る

似ている issue

DevOps の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。