Skip to content

ci(pr-agent): disable DeepSeek thinking for /improve fallback - #504

Merged
buke merged 3 commits into
mainfrom
ci/pr-agent-deepseek-reasoning
Oct 20, 2026
Merged

buke merged 3 commits into
mainfrom
ci/pr-agent-deepseek-reasoning

Conversation

@buke

@buke buke commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

User description

Summary

  • Set config.reasoning_effort: none so DeepSeek flash does not spend the output budget on CoT (fixes empty content with finish_reason=length on Gemini→DeepSeek fallback).
  • Raise config.ai_timeout from 180s to 300s for slower DeepSeek fallback calls.
  • Drop outdated comment that claimed reasoning_effort was a no-op for these models.

Test plan

  • After merge, trigger PR-Agent on a PR when Gemini is rate-limited (or temporarily force fallback) and confirm DeepSeek returns non-empty /improve suggestions.
  • Confirm workflow env still shows config.reasoning_effort: none in the Action log.
  • Spot-check a normal Gemini-primary run still completes review/describe/improve.

Summary by Sourcery

Improve PR-Agent reliability by disabling excessive model reasoning and refining timeout and fallback handling.

Bug Fixes:

  • Prevent PR-Agent fallback requests from exhausting their token budget in DeepSeek thinking mode and returning empty improvement content.
  • Ensure timed-out primary requests proceed to fallback models instead of retrying the same model.

Enhancements:

  • Configure PR-Agent with a bounded job timeout and a shorter per-call AI timeout to keep primary and fallback processing within the workflow limit.

CI:

  • Update the PR-Agent workflow configuration for reasoning effort, timeout handling, and fallback behavior.

PR Type

Enhancement


Description

  • Disable DeepSeek reasoning effort in PR-Agent.

  • Increase AI timeout for fallback requests.

  • Update configuration comments for PR-Agent workflow.


File Walkthrough

Relevant files
Configuration changes
pr-agent.yml
configure PR-Agent reasoning effort and timeout                   

.github/workflows/pr-agent.yml

  • Added config.reasoning_effort: "none" to prevent DeepSeek thinking
    mode from exhausting tokens.
  • Increased config.ai_timeout from 180s to 300s for fallback calls.
  • Removed outdated comments regarding reasoning_effort handling.
+7/-4     

PR-Agent injects reasoning_effort=medium (mapped to high on DeepSeek),
so CoT can hit max_tokens with empty content. Set reasoning_effort=none
and raise ai_timeout for the fallback path.
@sourcery-ai

sourcery-ai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The workflow now explicitly disables DeepSeek reasoning for structured /improve fallback responses and allows more time for slower fallback calls, addressing empty responses caused by token exhaustion while retaining the existing PR-Agent automation settings.

File-Level Changes

Change Details Files
Disable DeepSeek thinking and increase fallback timeout in the PR-Agent workflow.
  • Set config.reasoning_effort to none so fallback /improve responses preserve content instead of exhausting the output budget on chain-of-thought.
  • Increase config.ai_timeout from 180 to 300 seconds for slower DeepSeek fallback calls.
  • Replace the outdated reasoning-effort comment with documentation of the fallback behavior and configuration rationale.
.github/workflows/pr-agent.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Next included review available in 14 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 3 included reviews currently available. Your 41 included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: c5be352a-dd7a-4b8e-9969-ff2cd295b74f
📥 Commits

Reviewing files that changed from the base of the PR and between 1cd74eb and 5eab2d6.

📒 Files selected for processing: 1
  • .github/workflows/pr-agent.yml

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 57179c96-5a65-428f-976a-a6993b4cec5f
📥 Commits

Reviewing files that changed from the base of the PR and between f51e50a and 1cd74eb.

📒 Files selected for processing: 1
  • .github/workflows/pr-agent.yml

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.

📝 Walkthrough

The PR-Agent workflow sets DeepSeek reasoning effort to none and increases the AI timeout from 180 to 300 seconds. It also removes an obsolete comment about reasoning-effort support.

Changes: PR-Agent workflow configuration

Layer / File(s) Summary
Configure model settings
.github/workflows/pr-agent.yml
The workflow sets config.reasoning_effort to none, increases config.ai_timeout from 180 to 300 seconds, and removes an obsolete comment about reasoning-effort support.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 1cd74

This CI-only change disables DeepSeek reasoning and gives slower fallback calls more time. Gemini-primary requests remain valid. The main open question is whether the setting fully fixes empty /improve fallbacks, which the planned post-merge check will confirm. No blocking risk to existing review, describe, or improve runs was found.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly identifies the PR-Agent change: disabling DeepSeek thinking for the /improve fallback. It is concise and specific.
Description check Passed The description provides a clear summary, technical rationale, configuration changes, and a test plan. It does not repeat the CLA notice from the template, but the substantive PR information is comple…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description provides a clear summary, technical rationale, configuration changes, and a test plan. It does not repeat the CLA notice from the template, but the substantive PR information is complete.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Comment thread .github/workflows/pr-agent.yml Outdated
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path=".github/workflows/pr-agent.yml" line_range="51" />
<code_context>
+          # short structured /improve JSON. See https://api-docs.deepseek.com/zh-cn/guides/thinking_mode
+          config.reasoning_effort: "none"
+          # DeepSeek fallback often needs longer than Gemini free-tier calls.
+          config.ai_timeout: "300"
           # LiteLLM retries 429 with exponential backoff.
           NUM_RETRIES: "6"
</code_context>
<issue_to_address>
**Slow failures block PR-Agent runs**

When a provider is slow or unavailable and multiple PR-Agent events arrive while retries are in progress, `config.ai_timeout` applies to Gemini as well as DeepSeek, and `NUM_RETRIES: 6` can keep a slow or failing call occupying the sole `pr-agent` concurrency slot through repeated long waits. Other PR-Agent events are delayed, and GitHub may cancel an older pending run when a newer one arrives, so some reviews or suggestions never run.

Limit the longer timeout to fallback calls, or otherwise bound the total run duration so one failing provider cannot hold the shared concurrency slot for an extended period.
</issue_to_address>

### Comment 2
<location path=".github/workflows/pr-agent.yml" line_range="49" />
<code_context>
+          # reasoning_effort=medium (mapped to high). CoT can exhaust max_tokens
+          # and leave content empty (finish_reason=length). Disable thinking for
+          # short structured /improve JSON. See https://api-docs.deepseek.com/zh-cn/guides/thinking_mode
+          config.reasoning_effort: "none"
+          # DeepSeek fallback often needs longer than Gemini free-tier calls.
+          config.ai_timeout: "300"
</code_context>
<issue_to_address>
**Gemini reasoning is disabled**

When a run uses Gemini as its primary model and honors the configured reasoning effort, pR-Agent uses this shared setting for the configured Gemini primary as well as the DeepSeek fallback, so Gemini-primary review and improve calls run with reasoning disabled and deliver less reasoned results.

Apply `reasoning_effort: none` only to DeepSeek fallback requests, or preserve Gemini’s existing reasoning effort.
</issue_to_address>

Sourcery assessment

Approval pending. 2 findings to address first.

Blocking findings: .github/workflows/pr-agent.yml:51, .github/workflows/pr-agent.yml:49


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread .github/workflows/pr-agent.yml Outdated
Comment thread .github/workflows/pr-agent.yml
@github-actions

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Scope reasoning_effort to fallback model

reasoning_effort: "none" is applied globally, so it also affects the primary Gemini
model, not just the DeepSeek fallback. If Gemini rejects or ignores the value, this
could regress the primary path; consider scoping it to the fallback model or
verifying Gemini tolerates none.

.github/workflows/pr-agent.yml [49]

+          config.reasoning_effort: "none"
 
-
Suggestion importance[1-10]: 5

__

Why: The suggestion correctly notes that config.reasoning_effort applies globally and may affect the primary Gemini model, which is a legitimate concern given the removed comment explicitly stated it was a no-op for these models. However, it only asks to verify rather than providing a concrete fix, and the improved_code is identical to existing_code.

Low
Verify job timeout covers retries

Raising ai_timeout to 300s combined with NUM_RETRIES: "6" can make a single stuck
call block the job for a very long time (potentially exceeding the job timeout).
Confirm the job-level timeout-minutes accommodates the worst-case retry × timeout
product.

.github/workflows/pr-agent.yml [51]

+          config.ai_timeout: "300"
 
-
Suggestion importance[1-10]: 4

__

Why: The observation about ai_timeout: "300" combined with NUM_RETRIES: "6" potentially exceeding job timeout is a reasonable hardening note, but it is speculative and only asks for verification. The improved_code is identical to existing_code, offering no concrete change.

Low

buke added 2 commits October 20, 2026 10:04
Keep global reasoning_effort=none (PR-Agent has no DeepSeek-only knob).
Skip same-model timeout retries so Gemini failures fall through faster,
and add a job timeout-minutes ceiling for the shared concurrency slot.
Abandon slow PR-Agent runs early; lower ai_timeout so primary + fallback
fit inside the job ceiling.
@buke
buke merged commit fc03ded into main Oct 20, 2026
22 checks passed
@buke
buke deleted the ci/pr-agent-deepseek-reasoning branch October 20, 2026 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant