GLM-5.3 Cybersecurity Benchmarks: What 84.5% Means

GLM-5.3 scored 84.5% on CyberGym. That does not mean it can find 84.5% of the vulnerabilities in your codebase. Kill that translation now: the score comes from a demanding—but much narrower—test built around known, historical bugs.

Released by Z.ai on August 14, 2026, the model delivered the most interesting GLM-5.3 cybersecurity benchmarks yet. It leads the lab’s CyberGym comparison, more than doubles GLM-5.2 on an exploitation benchmark, and comes alongside a disclosure ledger tracking 2,436 claimed real-world findings. All three numbers matter. They do not measure the same thing.

Two lazy takes are competing here: “AI can hack everything” and “vendor benchmarks mean nothing.” Both dodge the interesting part. GLM-5.3 shows how much targeted post-training can extract from an unchanged base model—and why every cyber score needs four labels attached: task, harness, budget, and access.

Emergent does not mean accidental

The launch post opens with an unusually clean sentence: “Scaling post-training is all we did for GLM-5.3.” The base model is the same one behind GLM-5.2. Rather than paying for another pretraining run, the lab expanded its executable environments, diversified the tasks, and spent more compute teaching the existing model to operate inside them.

Cybersecurity was not a surprise elective the model discovered between calculus and poetry. The training mix explicitly added vulnerability-discovery data and environments. What surprised the researchers was how quickly that practice transferred from spotting isolated flaws to planning across several stages of exploitation.

That distinction makes “emergent cyber capability” defensible, but only with an asterisk. The capability’s rate and breadth were unexpected; its direction was not. Our GLM-5.2 API guide covered the predecessor as a long-context coding model. GLM-5.3 is a case study in changing the graduate program without rebuilding the student.

The result extends beyond security. Terminal-Bench 3.0 rose from 4.6 to 28.3, DeepSWE v1.1 climbed from 46.2 to 66.9, and AutomationBench went from 26.2 to 48.2. Those jumps support the broader post-training thesis. The cyber results tell us where that thesis becomes dual-use.

Those training environments are not glorified prompt files. The lab describes tasks with working codebases, compute, storage, documentation, hidden state, and verifiers that can decide whether the result works. A judge agent checks that each task is solvable; separate tests try to catch reward shortcuts. This is expensive scaffolding, but it teaches the model to finish an operation rather than merely suggest the next command.

What GLM-5.3 cybersecurity benchmarks actually test

The original CyberGym paper gives an agent a description of a historical vulnerability and the corresponding unpatched codebase. The agent must generate a proof-of-concept input that reaches the buggy code and triggers it. That requires repository-scale reasoning, tool use, and persistence. It does not require discovering that an unknown vulnerability exists.

ExploitGym’s task definition starts one step later and asks for something harder. The agent receives an input that already triggers a vulnerability, then must turn that foothold into concrete security impact such as unauthorized file access or code execution. Memory layout, runtime adaptation, and modern mitigations enter the picture. A crash is no longer enough.

Evidence	Starting point	Task	Does not prove
CyberGym	Bug description + vulnerable repo	Reproduce a known bug	Open-ended zero-day discovery rate
ExploitGym	Bug + triggering input	Build a working exploit	Full attack autonomy on live systems
Disclosure ledger	Real open-source projects	Find and disclose flaws	Coverage or false-positive rate without a denominator

The third row is not a benchmark at all. It is a real-world claim that depends on expert review and coordinated disclosure. Mixing the rows produces a wonderfully dramatic headline and a useless mental model.

Each stage also fails differently. A reproduction agent can miss the correct build path; an exploitation agent can trigger a crash without gaining control; a discovery system can drown maintainers in plausible-looking false positives. One percentage cannot describe all three failure modes. Security evaluation is a ladder, not a single high score painted on the wall.

There is genuine progress here. The best model-agent pairing in the 2025 CyberGym paper managed 11.9%; the lab now reports 84.5%. That is not a clean model-to-model comparison—the harnesses, models, and evaluation setup changed—but it shows why coding benchmarks often break in production. The system around a model can move almost as fast as the model itself.

The leaderboard win stops at the reproduction layer

The lab’s table puts GLM-5.3 first on CyberGym by less than one percentage point. The order flips when the task demands exploitation rather than reproduction. GLM-5.3’s ExploitBench score reaches 54.4%—a 30-point gain over GLM-5.2—but trails the two strongest closed-model results by more than 20 points.

Model result	CyberGym	ExploitBench	ExploitGym 2h / 6h
GLM-5.3	84.5%	54.4%	105 / 130
GLM-5.2	77.2%	24.4%	29 / 39
Mythos 5	83.8%	78.0%	181 / 247
GPT-5.6 Sol	83.6%	76.5%	216 / 293

ExploitGym tells the same two-sided story. GLM-5.3 completes 105 tasks under a normalized two-hour budget and 130 under six hours, up from 29 and 39. The stronger closed results reach 181/247 and 216/293. Capability is growing fastest where the safety stakes rise, while the absolute gap is also widest there.

The units deserve care. CyberGym reports a success percentage across 1,507 tasks. ExploitBench averages coverage over 41 tasks and three revisions. ExploitGym counts completed tasks from 869 instances after rescaling API time with a stated per-model throughput.

These are useful within-column comparisons, not ingredients for a blended “hacking score.” Benchmark soup remains soup, even when every number has three decimals.

So “state of the art at cybersecurity” is too broad. “First in this published CyberGym comparison” is accurate. SiliconANGLE’s release report captured the split: GLM-5.3 led CyberGym, fell behind on two other cyber tests, and entered a two-week wait for public weights.

The 2,436 real findings are the more important claim

Benchmarks are rehearsals. The Z.ai Security Disclosure Ledger is the attempt at a production record: 2,436 findings across 269 open-source projects, including 107 critical, 990 high, 1,286 medium, and 53 low-severity issues. Public examples affect Linux, WebKit, FreeBSD, GStreamer, Suricata, and Joomla.

The age distribution is as striking as the count. The ledger says the oldest flaw was introduced in 1981, while the average finding survived 26.6 years before discovery. Static analyzers have been scanning code for decades; an agent that can connect repository context, runtime behavior, and a valid trigger would be a materially different tool.

Coordinated disclosure slows the public proof by design. Maintainers need time to reproduce a report, build a patch, notify downstream users, and assign an identifier before technical details become a free tutorial for attackers. That process explains the embargo pile. It does not eliminate the need to audit the final acceptance rate once the pile starts clearing.

But the public evidence was thin on August 16. Only 53 findings—about 2.2%—were disclosed, while 2,383 remained under embargo. The ledger does not publish how many projects or lines were scanned without a finding, how false positives were counted, which model version found each bug, or what share survived expert review. A numerator this large deserves its denominator.

Security teams should treat these agents like ambitious junior researchers with shell access: useful, fast, and fenced in. Run them in isolated environments, restrict network and secret access, and require reproducible evidence before opening a report. Our OWASP AISVS evaluation guide offers a better starting scope than pointing a frontier model at production and hoping the permission dialog develops judgment.

The real test starts when the weights land

The 84.5% score describes more than GLM-5.3. It includes Claude Code 2.1.207 as the harness, maximum reasoning effort, up to 128,000 output tokens, restricted networking, and unlimited time per CyberGym task. ExploitGym uses throughput-normalized two- and six-hour budgets. Remove the harness, shrink the budget, or add real latency constraints, and you have a different product.

The lab said public weights would follow two weeks after August 14, after safety evaluation and hardening. That delay is sensible for a model trained to traverse exploit chains. It also means the central results remain vendor-run until outside teams can reproduce them.

The unresolved question is simple: can independent researchers match the cyber jump under the same harness and budgets—and how many of the 2,383 embargoed findings will survive disclosure review? The dangerous number is not 84.5%. It is the distance between a reproducible benchmark and an agent handed a real repository. If the schedule holds, the first answer should arrive around August 28, 2026, when the weights are due.