Skip to content

Retry failed benchmark result pushes #488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
savannahostrowski wants to merge 2 commits into
base: main
Choose a base branch
from
+41 −0
Open

Retry failed benchmark result pushes #488

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9ad27de
Retry failed pushes of benchmark results
savannahostrowski Nov 6, 2026
e8bc08a
Reuse Unix result push step with a YAML anchor
savannahostrowski Nov 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions bench_runner/templates/_benchmark.src.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,27 @@ jobs:
# just pulled in in that case.
git pull -s recursive -X ours --autostash --rebase
- name: Add data to repo
id: commit
uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
with:
add: results
push: false
- name: Push results with retry
if: ${{ steps.commit.outputs.committed == 'true' }}
shell: powershell
run: |
$branch = $env:GITHUB_REF_NAME
for ($attempt = 1; $attempt -le 5; $attempt++) {
Write-Host "Push attempt $attempt to origin/$branch"
git pull --rebase origin $branch
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
git push origin "HEAD:$branch"
if ($LASTEXITCODE -eq 0) { exit 0 }
if ($attempt -lt 5) {
Start-Sleep -Seconds (Get-Random -Minimum 10 -Maximum 40)
}
}
throw "Push failed after 5 attempts"
- name: Upload artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.0
with:
Expand Down Expand Up @@ -123,10 +141,30 @@ jobs:
# just pulled in in that case.
git pull -s recursive -X ours --autostash --rebase
- name: Adding data to repo
id: commit
if: ${{ !inputs.perf }}
uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
with:
add: results
push: false
- &push-results-unix
name: Push results with retry

@maurycy maurycy Nov 13, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to also cover _pystats, _generate and _find_failures?

maurycy@gimel bench_runner (macos-openssl3 9c6b42e?) % rg 'EndBug/add-and-commit'
bench_runner/templates/_benchmark.src.yml
86:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
127:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
167:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0

bench_runner/templates/_find_failures.src.yml
27:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0

bench_runner/templates/_generate.src.yml
49:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0

bench_runner/templates/_pystats.src.yml
64:        uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
maurycy@gimel bench_runner (macos-openssl3 9c6b42e?) %

If I understand correctly, they all compete for the same resource

if: ${{ steps.commit.outputs.committed == 'true' }}
shell: bash
run: |
branch="$GITHUB_REF_NAME"
for attempt in 1 2 3 4 5; do
echo "Push attempt $attempt to origin/$branch"
git pull --rebase origin "$branch"

@maurycy maurycy Nov 13, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want --autostash or -X ours here? Not sure if it's truly needed, just noted we've got different flags there:

https://github.com/python/bench_runner/pull/488/changes#diff-5b05116189731f1876572c843d755c125910b1039d317875831367f26293a310R203

if git push origin "HEAD:$branch"; then
exit 0
fi
if [ "$attempt" -lt 5 ]; then
sleep $(( RANDOM % 30 + 10 ))
fi
done
echo "Push failed after 5 attempts" >&2
exit 1
- name: Upload benchmark artifacts
if: ${{ !inputs.perf }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.0
Expand Down Expand Up @@ -164,9 +202,12 @@ jobs:
# just pulled in in that case.
git pull -s recursive -X ours --autostash --rebase
- name: Add data to repo
id: commit
uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
with:
add: results
push: false
- *push-results-unix
- name: Upload artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.0
with:
Expand Down
Loading