Too much updates means no updates :( Windows Server Update Service Sync Operations Issues and Timeouts

TL;DR

If your host is unable to retrieve metadata for their approved updates in WSUS, resulting in no updates available, you may experience the same issue I describe here.

This entry is about the gap between "you know nothing Jon Snow" and "you are up to date" while automating Windows patching - in my case for AWS AppStream images (WorkSpaces Applications, as AWS now calls them) with AWX and an on-prem WSUS server.

You, the reader, also need to patch up, so one day it may catch up with you.

My patching workflow

Based on my Excel schedule sheet, which also works as my lightweight CMDB, my workflow job checks every day whether there is planned patching for a certain fleet. Then it creates a job schedule if there is none, or skips it if there is one. For example, as of today, 9th October, it checks the closest patching date for FLEET1, takes its name, time and date, and checks whether a patching job is already planned.

My AppStream fleets do not get updated in place. The plan was a chain of job templates in one workflow:

First, I look for the right fleet, take its image and start an image builder from it – by duplication.
Reset SusClientId – otherwise WSUS considers the host an old client, even though the hostname has changed.
Point the builder at WSUS, move it into the right WSUS group and install everything approved.
Verify that nothing is left to install.
Create a new image from the builder.
Swap the image in the fleet.

Every step passes its results to the next one as workflow artifacts. The builder is reached over PSRP, as is WSUS, and AWS through a cross-account role. Seems simple. After a successful patching operation and image change, the image builder is shut down, for cost-saving purposes ofc.

Zero updates, job green

Then came a run that looked perfect:

TASK [Search for available updates (post-delay)]
    "found_update_count": 0,
    "installed_update_count": 0,

TASK [Final success message]
    "msg": "Patch run completed (found=0, installed=0)"

PLAY RECAP
localhost : ok=28 changed=7 unreachable=0 failed=0

The image got published. I asked the WSUS admin to look at the host, and the next morning the console showed this:

Last Status Reported:  Not yet reported
0 updates failed to install
0 updates have not been installed
0 updates have been installed or are not applicable
5625 updates have unknown status

WSUS had never heard a word from that client. Further up the same job log there was a line I had happily ignored. An early diagnostic scan had thrown 0x80244010, and the task had failed_when: false on it. So I started debugging. I connected to the image builder (I can, because I leave them in place for rollback – just shut down, without deleting the latest one) and ran Get-HotFix, which showed the September cumulative update. That's just fine, but a pipeline that cannot tell "nothing to install" from "I never got any answer from WSUS, so I have no clue" will end up publishing an unpatched image with a green tick on it – a false positive.

The group you ask for is not the group you get

To check a builder independently, I wrote a read-only verification playbook. It scans from the client, asks the WSUS API what the server thinks of the same host, and compares the two. On a builder started from the new image, its first clean run said this:

"PendingCount": 0,
"TargetGroup": "appstream",
"TargetGroupEnabled": 1,
...
"Groups": ["Unassigned Computers", "All Computers"],
"RequestedGroup": "APPSTREAM",
"TargetingMode": "Server"

The client asks to be in appstream, and WSUS ignores it. With TargetingMode set to Server, the TargetGroup value in the registry is decoration. Group membership is whatever someone set on the server, and a new client lands in Unassigned Computers. So that PendingCount: 0 was a scan against the approvals for Unassigned Computers, not the 4,643 updates approved for appstream. It measured nothing I cared about.

The patch playbook already moved the host from Unassigned into the target group, which is why patching mostly worked. Verification did not, and it still answered with a confident zero.

Moving the host is only half of it. After the move, the client has to scan again and then send a status report before the server knows anything. Reporting is asynchronous. If the builder is stopped for imaging a few minutes after the run, the report may simply never leave.

And now we are getting to the bottom of the problem. After moving to the correct group in WSUS, the client has to scan again: it downloads the update metadata, compares it with what WSUS has approved for its group, and installs what is missing. Only then does it REPORT its status back. Until that report arrives, WSUS knows nothing about the client, and neither do I.

0x80244010, or too much metadata at once

With the verification playbook retrying the client scan, I ran a check:

"ScanAttempts": [
    "#1: 0x80244010",
    "#2: OK"
]

0x80244010 is WU_E_PT_EXCEEDED_MAX_SERVER_TRIPS. The Windows Update agent pulls update metadata from WSUS in chunks, and WSUS caps each chunk with MaxXMLPerRequest, 5 MB by default. The client gives up after 200 round trips. The old WSUS team blog puts it bluntly: the limit is defined in code and cannot be reconfigured. If a client needs more than 200 chunks of metadata on its first sync, it fails. The next attempt resumes from what is already cached and usually gets through.

That explained the "fails once, works on retry" behaviour. It also showed that I had been making it worse myself. The patch playbook starts with a "WSUS full reset": stop the services, wipe SoftwareDistribution, delete SusClientId. That throws away the cache that would have let the second attempt succeed. Every new builder started its first sync from nothing, against a server with a lot of metadata to hand out.

It still did not explain why there was so much metadata. Thousands of approvals contribute, but a builder sitting in Unassigned Computers hit the same error, and that group should have far fewer approvals.

14,466 detectoids nobody asked for

The answer was a Microsoft known issue from July 2026, KB5121986. From mid-July, a large number of test detectoids were published to the WSUS channel by mistake, with titles like "Product Detectoid for ProductName TestProduct". Detectoids are metadata that every client downloads, whatever is approved. Any WSUS server that synced during that window keeps them in SUSDB. The KB lists 0x80244010 as the key symptom and includes Windows Server 2019, which is exactly what AppStream builders run.

Microsoft fixed the source on 18 July. Servers that had already pulled the junk have to be cleaned locally. The KB ships a SQL script that deletes the detectoids, lifts MaxXMLPerRequest while clients resync, and then restores it.

I am not the WSUS admin, and "I think your server has a bug" is not a strong opening. So before asking anyone to touch the database, I ran a read-only count using the same filter as the KB script:

$setup  = Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Update Services\Server\Setup'
$server = if ($setup.SqlServerName -match 'MICROSOFT##WID') {
            '\\.\pipe\MICROSOFT##WID\tsql\query' } else { $setup.SqlServerName }
$cn = New-Object System.Data.SqlClient.SqlConnection "Server=$server;Database=SUSDB;Integrated Security=True"
$cn.Open()
# SELECT COUNT(*) ... WHERE p.UpdateType = 'Detectoid'
#   AND tlp.Title LIKE 'Product Detectoid for ProductName TestProduct%'

My result:

SQL: MICROSOFT##WID  DB: SUSDB
TestDetectoids   = 14466
MaxXMLPerRequest = 5242880

There should be zero of those, and we had 14,466. MaxXMLPerRequest still at its default showed that the KB procedure had never been run. That went to the WSUS admin with the KB link and the two numbers, which is a much easier conversation than "the clients feel slow".

What the pipeline checks now

The cleanup belongs to the WSUS admin, but in my LAB environment I got 14,169, and after applying the cleanup from the KB it dropped to 0.

So I changed the pipeline to stop believing a zero it cannot back up. It now has two gates before it is allowed to search and install: the client scan must actually succeed, retried rather than swallowed when it hits 0x80244010, and WSUS must have received a status report from the builder. No successful scan, or no report, means no image.

If the builder is not in the target group, or the scan failed and WSUS has no report, the pending count is now unknown rather than 0. With fail_on_pending set, unknown fails the job exactly as an outstanding update would.

Two smaller lessons came out of the debugging. win_powershell serialises output only two levels deep by default, so nested objects from the WSUS API arrive as strings like @{Id=...; Groups=System.Object[]} unless you set depth. And a job template running the wrong playbook fails very quickly when that playbook checks its inputs first, which is the best way to find out.

Zero needs a witness

A metadata limit and a bad batch of detectoids from upstream produced a result that looked like success.

My takeaway is to make the server confirm it has heard from the machine, and to make "I don't know" a separate state from "nothing to do".

Koperun

Search This Blog