The PR quality score
An integer from 0 to 100, or nothing at all. It measures how many known low-effort patterns a pull request trips, and nothing else.
How the number is produced
Every heuristic is a detector for something bad. Passing is the normal case, so the score starts at 100 and comes down.
- Start at a ceiling of 100.
- For each blocker that fired, lower the ceiling. One blocker caps the score at 50, two at 35, three or more at 20.
- A few heuristics carry their own cap. A trivial patch caps the score at 50, or at 25 if the title is also vague or the body is empty. The lowest cap wins.
- For every other heuristic that fired, subtract ten points per weight. Weight 1 costs 10, weight 2 costs 20, weight 3 costs 30.
- Some heuristics compute their own penalty instead, so it scales with what they measured: one point per inline code reference over the limit, five points per commit that is not in Conventional Commits form.
- Floor at 0.
const W4_SCORE_CAPS = [50, 35, 20];
const PENALTY_PER_WEIGHT = 10;
ceiling = min(perSignalCaps..., w4Cap(blockersFired))
score = max(0, ceiling - deductions)A heuristic that is turned off is not counted. A heuristic that has never run against this pull request is not counted either. If nothing ran, there is no score rather than a score of zero.
Four examples
| What fired | Arithmetic | Score |
|---|---|---|
| Source added with no tests (w2), one overlong commit message (w1) | 100 − 20 − 10 | 70 |
| One AI watermark phrase (blocker), nothing else | ceiling 50, no deductions | 50 |
| AI watermark and honeypot hit (two blockers), plus emoji count (w1) | ceiling 35 − 10 | 25 |
| All three blockers, plus emoji count (w1) | ceiling 20 − 10 | 10 |
With the default comment threshold of 20, the third example is silent and the fourth gets a public warning comment.
The four weights
Weights are fixed in the code. Maintainers can turn a heuristic off or change its threshold. They cannot change what it costs.
| Weight | Meaning | Effect |
|---|---|---|
| 1 | Mild | −10 |
| 2 | Major | −20 |
| 3 | Critical | −30 |
| 4 | Blocker | caps the score at 50, 35 or 20 |
Blocker is reserved for 3 things that are close to proof rather than suspicion: a phrase a language model left in the body, honeypot text copied out of the pull request template, and ignoring the template altogether. Passing everything else should not rescue a pull request that did one of those.
Every heuristic
36 heuristics in six groups, 29 of them on by default. The identifiers are the exact keys stored on each scored pull request, so a maintainer looking at raw signals can match them one to one.
Size
size · 4 heuristics · 4 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Mega PR (huge diff in a single commit)size.mega_pr | 3 critical | on50 | Trips when both file and line counts exceed thresholds AND commit count is 1: a typical AI-bulk-generation pattern. |
Trivial patch (too small to score high)size.trivial_patch | 3 critical | on3 | PR changes ≤ N lines in ≤ 1 file with ≤ 1 commit: likely a typo, default-web-UI edit, or probe PR. Caps the score at 50% (or 25% when paired with a vague title or empty body). |
Excessive file countsize.file_count | 2 major | on50 | Penalize unfocused PRs touching too many files. |
Excessive line countsize.line_count | 2 major | on10000 | Penalize PRs with very large diffs. |
PR text
pr · 11 heuristics · 9 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Honeypot keyword hitpr.honeypot_hit | 4 blocker | on | PR body contains hidden honeypot text from the project's PR template, typically copy-pasted by AI bots. |
PR doesn't use the repo's PR templatepr.uses_template | 4 blocker | on0 | Repo ships a PR template (e.g. .github/PULL_REQUEST_TEMPLATE.md). When the template has checklist items, the body must include them. Threshold sets how many checkboxes may be missing (default 0). Match strictness is set project-wide (Quality core settings, default 80%). When the template has no checkboxes, at least one heading must appear. Skipped when the repo has no template. |
AI watermark phrasepr.ai_watermark | 4 blocker | on | Body contains a phrase commonly emitted by language models (e.g. "as an AI", "Here is the updated"). |
Body adds too many extra headers beyond the templatepr.template_extra_headers | 3 critical | on0 | Counts headers in the body whose text doesn't appear in the repo's PR template. Threshold is the maximum number of extra headers admins allow (default 0). Skipped when the repo has no template. |
Empty PR bodypr.body_empty | 2 major | on | Body is missing or whitespace-only. |
Vague PR titlepr.title_vague | 2 major | on | Title is a single vague word (update / fix / wip / patch / changes / misc / stuff / chore), GitHub's web-UI default ("Update README.md"), under 8 chars, or emoji-only. |
AI description assessmentpr.ai_assessment | 2 major | off40 | Fires when a model judges the PR's description to be a poor guide to what the change actually does. Only applies to PRs where the AI assessment task has been run; PRs without one are unaffected. |
Wall-of-text PR bodypr.body_too_long | 1 mild | on2500 | Body exceeds the configured character limit. |
Excessive emojis in bodypr.body_emoji_count | 1 mild | on2 | Many AI-generated PRs are decorated with emojis. |
Excessive inline code referencespr.body_inline_code_refs | 1 mild | on5 | Walls of inline ` ` references often indicate AI summaries. |
No linked issuepr.no_linked_issue | 1 mild | off | Body has no #N reference or fixes/closes/resolves keyword. |
Commits
commit · 5 heuristics · 4 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Author mismatch across commitscommit.author_mismatch | 2 major | on | Commits authored by multiple identities, usually cherry-picked work. Threshold is an allowlist of author emails or logins (one per line) that don't count toward the distinct-author tally (e.g. bot accounts like dependabot[bot] or noreply@github.com). |
Single giant commitcommit.single_giant | 2 major | on2000 | One commit with very many changes: typical AI bulk diff. |
Commit message too longcommit.message_too_long | 1 mild | on500 | Any single commit message exceeds the configured length. |
Conventional Commits formatcommit.conv_commits | 1 mild | off | Each commit message must match `type(scope): subject`. Off by default. |
Whitespace-only commitscommit.whitespace_only | 1 mild | on | At least one commit changes only whitespace (heuristic: tiny diffs only modify ws characters in the patch). |
Code
code · 8 heuristics · 8 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Lockfile-only PRcode.lockfile_only | 3 critical | on | PR changes only lockfiles: almost always low-effort. |
Excessive added commentscode.excessive_added_comments | 2 major | on10 | Ratio of added comment lines to added code lines is high: typical AI over-commenting. |
Touches blocked pathscode.blocked_paths | 2 major | on**/package-lock.json, **/pnpm-lock.yaml, **/yarn.lock, **/Cargo.lock, **/Gemfile.lock, **/go.sum, **/poetry.lock, **/composer.lock, dist/**, build/**, out/**, **/*.min.js, **/*.min.css | PR modifies paths listed in the project's blocked-paths config (default: lockfiles, dist, build, minified files). |
Code added without testscode.test_to_code_ratio | 2 major | on | Adds source files but no test files are added or modified. |
Formatter-only diffcode.formatter_only | 2 major | on | Diff appears to be exclusively reformatting (every hunk has equal token counts ignoring whitespace). |
Binary or generated files addedcode.binary_or_generated | 2 major | on | Adds binary files or files matching common generated patterns (*.min.js, *.lock, dist/**). |
Missing final newlinecode.missing_final_newline | 1 mild | on | Any added file missing a trailing newline character. |
Docs-only changescode.docs_only_in_code_repo | 1 mild | on | Only Markdown files changed. |
Diff cohesion
diff · 2 heuristics · 1 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Cross-module sprawldiff.cross_module | 1 mild | off4 | PR spans more than the configured number of top-level directories. |
Suspicious renamesdiff.suspicious_renames | 1 mild | on10 | Many file renames in a single PR: often AI bulk-rename slop. |
Account
account · 6 heuristics · 3 on by default| Heuristic | Weight | Default | What it looks for |
|---|---|---|---|
Mass forkingaccount.mass_forking | 3 critical | off6 | Author has created an unusually large number of forks in the last 24 hours. |
Spam-like usernameaccount.spam_username | 2 major | on | Username matches common spam/AI patterns (digit-heavy, ai/gpt/bot suffix, etc.). |
Account too newaccount.too_new | 2 major | on30 | GitHub account created less than the configured number of days ago. |
Low global merge ratioaccount.low_merge_ratio | 2 major | off30 | Author's global merged-PR / total-PR ratio is below the configured percent. |
Thin profileaccount.profile_thin | 2 major | on | Empty bio AND no avatar AND <2 followers AND <1 public repo. Often a throwaway account. |
No public emailaccount.no_email | 1 mild | off | Profile exposes no public email address. |
What maintainers can change
Per project
- Whether scoring runs at all. Off by default.
- Which heuristics are enabled.
- Each heuristic's threshold.
- The honeypot phrases.
- Template match strictness, default 80 percent.
- The score below which a public comment is posted, default 20.
Fixed in the code
- The weights.
- The blocker caps.
- The ten points per weight.
- What each heuristic actually looks for.
Changing the configuration takes effect immediately, including on pull requests that were scored last month. Raw signals are stored per pull request and the score is recomputed from them on every read, so there is no recompute job and no stale number.
Who sees the score
Maintainers, in the dashboard. That is the default and usually the whole story.
The one public surface is the warning comment, posted at most once per pull request when the score is under the project's threshold. It names the score and lists which heuristics flagged, with their measured values.
What it does not do
It does not read your code for correctness. It does not decide whether a change is a good idea. It does not block a merge, and it is not wired into branch protection.
It flags patterns that correlate with low-effort and machine-generated pull requests. That is all it is: correlation. A careful one-line fix to a typo trips the trivial patch heuristic and scores badly. A sloppy thousand-line pull request with a filled-in template can score well. The number is triage, not judgement.