Skip to content

Give CLI-initiated Solr connections short, shared connect/idle timeouts #4955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
epugh wants to merge 5 commits into from
Closed

Give CLI-initiated Solr connections short, shared connect/idle timeouts #4955

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b96a289
Give CLI-initiated Solr connections short, shared connect/idle timeouts
epugh Sep 27, 2026
542cad3
Merge remote-tracking branch 'upstream/main' into SOLR-cli-configurab…
epugh Oct 7, 2026
86f4dce
shorten up the connect timeout
epugh Oct 7, 2026
17ce135
fix format
epugh Oct 7, 2026
5dc0cb4
Merge branch 'main' into SOLR-cli-configurable-timeouts
jaykay12 Oct 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions changelog/unreleased/cli-configurable-timeouts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc
title: bin/solr CLI commands now fail fast (3s connect / 30s idle) against an unresponsive
node instead of inheriting SolrJ's 60s/600s bulk-indexing defaults.
type: changed
authors:
- name: Eric Pugh
links:
- name: PR#4955
url: https://github.com/apache/solr/pull/4955
18 changes: 17 additions & 1 deletion solr/core/src/java/org/apache/solr/cli/CLIUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,16 @@ private CLIUtils() {}
private static final long MAX_WAIT_FOR_CORE_LOAD_NANOS =
TimeUnit.NANOSECONDS.convert(1, TimeUnit.MINUTES);

/**
* Connect and idle timeouts used for CLI-initiated Solr connections. These are much shorter than
* {@link org.apache.solr.client.solrj.impl.SolrHttpConstants}' defaults, which are sized for
* long-running requests like bulk indexing or replica recovery; a CLI command run by a human at a
* terminal should fail fast instead of hanging for minutes against an unresponsive node.
*/
public static final int CLI_CONNECTION_TIMEOUT_SECONDS = 3;

public static final int CLI_IDLE_TIMEOUT_SECONDS = 30;

@jaykay12 jaykay12 Oct 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My local review agent was against this idea:

Here's what it quoted to me.

Shortening idle to 30s is not equivalent to “fail fast on a dead node,” because in SolrJ withRequestTimeout falls back to the idle timeout when unset (HttpSolrClient.Builder.getRequestTimeoutMillis() → getIdleTimeoutMillis()). Jetty then applies that value as the total request timeout (req.timeout(...)), not only as a quiet-socket limit.

So withIdleTimeout(30, SECONDS) without an explicit longer withRequestTimeout caps every HTTP round-trip at 30s wall-clock. That replaces SolrJ’s 600s default and will abort legitimate long CLI work that still has healthy I/O (export, streaming expressions, postlogs, large posts, package install, snapshot export, etc.).

Fail-fast against an unreachable node is a connect timeout concern — 3s there is right. Leave idle/request at SolrJ defaults (600s) so hung-connect fails quickly while long-running commands keep working. If we later want a short idle for “no bytes for N seconds,” we must set withRequestTimeout to DEFAULT_SO_TIMEOUT (or similar) explicitly so request duration is not coupled to idle.


private static CloudSolrClient.CloudSolrClientConnection resolveSolrConnectionFromCli(
CommandLine cli) throws IOException {
String solrConnection =
Expand Down Expand Up @@ -146,6 +156,8 @@ public static SolrClient getSolrClient(String solrUrl, String credentials, boole
var builder =
new HttpJettySolrClient.Builder(solrUrl)
.withMaxConnectionsPerHost(32)
.withIdleTimeout(CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withKeyStoreReloadInterval(-1, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(credentials);

Expand Down Expand Up @@ -300,7 +312,11 @@ public static String solrUrlFromConnection(
if (!solrConnection.isZookeeper()) {
return normalizeSolrUrl(solrConnection.quorumItems().get(0), false);
}
var builder = new HttpJettySolrClient.Builder().withOptionalBasicAuthCredentials(credentials);
var builder =
new HttpJettySolrClient.Builder()
.withIdleTimeout(CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(credentials);
Comment on lines +316 to +319

@jaykay12 jaykay12 Oct 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any merits in extracting this HttpJettySolrClient Builder into a common class say CLIUtils & using it in all these 4 tools?

would make the code duplication little lesser.

try (CloudSolrClient cloudSolrClient = getCloudSolrClient(solrConnection, builder)) {
Set<String> liveNodes = cloudSolrClient.getClusterState().getLiveNodes();
if (liveNodes.isEmpty())
Expand Down
6 changes: 5 additions & 1 deletion solr/core/src/java/org/apache/solr/cli/ExportTool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,11 @@ DocsSink getSink() {
abstract void exportDocs() throws Exception;

void fetchUniqueKey() throws SolrServerException, IOException {
var builder = new HttpJettySolrClient.Builder().withOptionalBasicAuthCredentials(credentials);
var builder =
new HttpJettySolrClient.Builder()
.withIdleTimeout(CLIUtils.CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLIUtils.CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(credentials);

solrClient =
new CloudSolrClient.Builder(List.of(baseurl)).withHttpClientBuilder(builder).build();
Expand Down
6 changes: 5 additions & 1 deletion solr/core/src/java/org/apache/solr/cli/HealthcheckTool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.Option;
import org.apache.commons.cli.Options;
Expand Down Expand Up @@ -129,7 +130,10 @@ public void runImpl(CommandLine cli) throws Exception {
cli.getOptionValue(COLLECTION_NAME_OPTION),
cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION));
var builder =
new HttpJettySolrClient.Builder().withOptionalBasicAuthCredentials(params.credentials());
new HttpJettySolrClient.Builder()
.withIdleTimeout(CLIUtils.CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLIUtils.CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(params.credentials());
try (var cloudSolrClient = CLIUtils.getCloudSolrClient(solrConnection, builder)) {
echoIfVerbose("Connecting to Solr at " + solrConnection.toString());
runCloudTool(cloudSolrClient, params);
Expand Down
2 changes: 2 additions & 0 deletions solr/core/src/java/org/apache/solr/cli/PostLogsTool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ public void runCommand(String baseUrl, String root, String credentials) throws I
var builder =
new HttpJettySolrClient.Builder(URLUtil.extractBaseUrl(baseUrl))
.withDefaultCollection(URLUtil.extractCoreFromCoreUrl(baseUrl))
.withIdleTimeout(CLIUtils.CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLIUtils.CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withKeyStoreReloadInterval(-1, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(credentials);
try (SolrClient client = builder.build()) {
Expand Down
6 changes: 5 additions & 1 deletion solr/core/src/java/org/apache/solr/cli/StreamTool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.Option;
import org.apache.commons.cli.Options;
Expand Down Expand Up @@ -426,7 +427,10 @@ void runStream(
private StreamContext createStreamContext(
CloudSolrClient.CloudSolrClientConnection solrConnection, String credentials) {
var jettyClientBuilder = new HttpJettySolrClient.Builder();
jettyClientBuilder.withOptionalBasicAuthCredentials(credentials);
jettyClientBuilder
.withIdleTimeout(CLIUtils.CLI_IDLE_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withConnectionTimeout(CLIUtils.CLI_CONNECTION_TIMEOUT_SECONDS, TimeUnit.SECONDS)
.withOptionalBasicAuthCredentials(credentials);
HttpJettySolrClient client = jettyClientBuilder.build();

// subclass so we can ensure our client is closed when the cache is closed
Expand Down
Loading