mem-check: CI's start-up PSS has crept up to the 165 MB baseline, so the gate fails on noise #96

New issue

Closed

opened 2026-10-27 22:49:07 +00:00 by claude-bot · 2 comments

claude-bot commented

2026-10-27 22:49:07 +00:00

Collaborator

tools/mem-check/budget-kb (165000) was set on 2026-10-20 at about 10% over the measured 143–149 MB (docs/09). CI's peaks on the last app jobs, on commits that don't touch start-up:

job	normal	slow
1701	159,961	163,887
1711	158,137	156,439
1716	163,014	—
1727	154,581	156,306
1745	162,397	162,885
1757 (PR #90, `97f46c2`)	165,056 FAIL	—

So the spread is ~10 MB and the mean has drifted to ~160 MB: the headroom is gone and PR #90 failed by 56 kB on a change that adds two 96 px PNGs to the qrc (start-up doesn't load the page that uses them). Locally under load the same binary measured 165–180 MB, and an interleaved A/B with the base QML showed the same spread.

To do:

Find what raised the start-up peak from ~146 to ~160 MB since 10-20 (bisect with mem-check on the merged PRs; candidates: the footer menu/icons, the Bible switcher, translations loading, the runner's image). If it's real, it should show on the phone too. The phone peaked at 79.9 MB with Modules open (PR #89) against the 80 MB device budget, so that's worth knowing.
Make the gate robust to noise: take the min (or median) of 3 runs per profile instead of one, and re-set the baseline ~10% over the new median. A single sample on a shared runner is what makes this flaky.

Should: a flaky trunk gate costs a 10+ min rerun each time on the single runner.

`tools/mem-check/budget-kb` (165000) was set on 2026-10-20 at about 10% over the measured 143–149 MB (docs/09). CI's peaks on the last app jobs, on commits that don't touch start-up: | job | normal | slow | |---|---|---| | 1701 | 159,961 | 163,887 | | 1711 | 158,137 | 156,439 | | 1716 | 163,014 | — | | 1727 | 154,581 | 156,306 | | 1745 | 162,397 | 162,885 | | 1757 (PR #90, 97f46c2) | **165,056 FAIL** | — | So the spread is ~10 MB and the mean has drifted to ~160 MB: the headroom is gone and PR #90 failed by 56 kB on a change that adds two 96 px PNGs to the qrc (start-up doesn't load the page that uses them). Locally under load the same binary measured 165–180 MB, and an interleaved A/B with the base QML showed the same spread. To do: 1. Find what raised the start-up peak from ~146 to ~160 MB since 10-20 (bisect with mem-check on the merged PRs; candidates: the footer menu/icons, the Bible switcher, translations loading, the runner's image). If it's real, it should show on the phone too. The phone peaked at 79.9 MB with Modules open (PR #89) against the 80 MB device budget, so that's worth knowing. 2. Make the gate robust to noise: take the min (or median) of 3 runs per profile instead of one, and re-set the baseline ~10% over the new median. A single sample on a shared runner is what makes this flaky. Should: a flaky trunk gate costs a 10+ min rerun each time on the single runner.

claude-bot added the

priority

Should

label

2026-10-27 22:49:07 +00:00

claude-bot referenced this issue

2026-10-28 10:49:05 +00:00

e2e typed-reference flaky: no 'utsb: menu settings' after 10 s #98

claude-bot referenced this issue

2026-10-28 10:53:09 +00:00

F1: starter sets and module manager #10

claude-bot referenced this issue

2026-10-28 13:09:15 +00:00

Shared QML controls in qml/controls/ (#91) #102

claude-bot referenced this issue

2026-10-28 13:15:31 +00:00

Shared QML controls in qml/controls/ (#91) #102

claude-bot commented

2026-10-28 15:37:10 +00:00

Author

Collaborator

Another one: run 183 (the ci-preview/107 push of 4e0d501, the same tree as PR #108's head, whose PR run 181 passed). The app job failed in the slow profile at peak PSS 165922 kB, 922 kB over the 165000 kB budget. The normal profile was at 160508 kB. I re-ran it.

Another one: run 183 (the `ci-preview/107` push of 4e0d501, the same tree as PR #108's head, whose PR run 181 passed). The `app` job failed in the slow profile at peak PSS 165922 kB, 922 kB over the 165000 kB budget. The normal profile was at 160508 kB. I re-ran it.

claude-bot commented

2026-10-28 17:53:38 +00:00

Author

Collaborator

And again: run 188 (PR #108 at 98d88fa, a docs-only change on a tree that passed) failed in the normal profile at 165092 kB, 92 kB over the budget. That makes two of the last three app runs failing on noise, so this is now holding up merges. It may be worth moving it up the queue.

And again: run 188 (PR #108 at 98d88fa, a docs-only change on a tree that passed) failed in the normal profile at 165092 kB, 92 kB over the budget. That makes two of the last three `app` runs failing on noise, so this is now holding up merges. It may be worth moving it up the queue.

claude-bot referenced this issue

2026-10-28 18:17:36 +00:00

CI: skip main's repeat of a passed PR tree; cancel superseded runs (#107) #108

claude-bot referenced this issue

2026-10-28 18:18:38 +00:00

CI: stop main-branch runs from holding up PR runs #107