Skip to content

feat: add OpenAI tool calling to Responses, Chat, and Prompt - #2626

Open
Rana Singh (ranadeepsingh) wants to merge 23 commits into
microsoft:masterfrom
ranadeepsingh:feat/openai-responses-tool-calling
Open

feat: add OpenAI tool calling to Responses, Chat, and Prompt#2626
Rana Singh (ranadeepsingh) wants to merge 23 commits into
microsoft:masterfrom
ranadeepsingh:feat/openai-responses-tool-calling

Conversation

@ranadeepsingh

@ranadeepsingh Rana Singh (ranadeepsingh) commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

What changed

This PR adds function calling to OpenAIResponses,
OpenAIChatCompletion, and both OpenAI modes of OpenAIPrompt.

Users can set tools, tool_choice, and parallel_tool_calls once or from a
DataFrame column. Responses also supports max_tool_calls. All three
transformers write function calls to the same structured toolCallsCol.

SynapseML carries function definitions, model calls, and tool results. It never
runs the application function. User code must validate the model-generated
name and arguments, perform the authorized work, and return one result for each
call ID.

Endpoint mapping

Behavior Chat Completions Responses
Function definition Nested tools[].function Flat function tool
Model call choices[].message.tool_calls[] output[] item with type: "function_call"
Correlation key tool_calls[].id function_call.call_id
Tool result role: "tool" and tool_call_id type: "function_call_output" and call_id
Stored continuation Replay message history Use previous_response_id or conversation
Stateless continuation Replay messages Replay input and prior output items

The implementation also adds:

  • typed Responses continuations and Chat assistant/tool continuation messages;
  • scalar and column helpers in the generated Python API;
  • strict recursive function-schema validation;
  • row-level errors for malformed column values;
  • null, empty, collision, and non-finite number validation;
  • local HTTP integration tests; and
  • DataFrame examples and tool-calling notebooks.

For strict: true, each object schema must set
additionalProperties: false, list every property in required, and avoid
undefined names in required. SynapseML warns when Azure strict schemas use
parallel calls, but it does not rewrite the caller's request.

Behavior and limits

  • Existing text-first OpenAIPrompt behavior and default Chat mode are
    unchanged.
  • Public JVM response case-class constructor shapes are unchanged.
  • Responses-only helpers fail clearly when called in Chat mode.
  • Typed Responses tool outputs currently accept strings. Image and file output
    arrays require raw input items.
  • Provider tools can pass through, but typed projection handles function calls.
  • Spark HTTP execution is at least once. Paid calls should be materialized, and
    side effects should be idempotent by call ID.

What to review

Reviewers should check:

  1. Chat and Responses wire payloads against their different endpoint formats;
  2. stored and stateless continuation correlation by call ID;
  3. strict-schema and row-level validation without changing caller payloads;
  4. compatibility of public JVM schemas and existing Prompt behavior; and
  5. generated Python methods and notebook examples.

How was this patch validated?

  • 60 tests passed across the offline tool-calling suites, including Chat and
    Responses local HTTP integrations.
  • 25 multimodal tests passed.
  • Cognitive main and test compilation plus Scalastyle passed.
  • Spark 4.1 test compilation passed on the release branch.
  • All 72 release-pipeline configuration tests passed.
  • Code generation and the focused generated-wrapper Python tests passed.
  • Pinned Black left all checked Python files unchanged.

Azure build
234443821
published 77 green checks for merge commit
1aa32d2427f96b194d8a5cba1ecd3873dc5afa64 and exact PR head
7a1f7907c0eb20bebbc37e2e141453efa8d1acf6. Azure labels the aggregate
partiallySucceeded because non-gating dependency-cache telemetry hit the
known certificate-name mismatch. No product job failed. All review threads are
resolved, and the current-head Copilot review has no findings.

Documentation

Does this PR change any dependencies?

  • No.
  • Yes.

Does this PR add a user-facing feature?

  • No.
  • Yes. The PR includes tool-calling notebooks and generated Python
    examples.

Copilot AI lite review requested due to automatic review settings August 12, 2026 20:30
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

Copy link
Copy Markdown

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends SynapseML’s OpenAI integrations to support the OpenAI Responses API tool/function-calling workflow end-to-end (tool declarations, structured tool calls, and typed continuation outputs), while expanding support for newer Responses parameters and improving schema/DTO compatibility.

Changes:

  • Adds tool normalization/validation utilities and Spark column helpers for tool calls, replay items, and typed function_call_output continuations.
  • Extends OpenAIResponses and OpenAIPrompt to support scalar + per-row tool configuration, modern Responses fields, and compatibility-preserving response parsing (via an internal V2 schema).
  • Adds Scala + Python tests (offline, integration, and opt-in live) plus documentation notebooks illustrating safe, explicit DataFrame workflows.
Show a summary per file
File Description
docs/Explore Algorithms/OpenAI/Quickstart - OpenAI Responses Tool Calling.ipynb New notebook documenting a two-turn tool-calling + continuation Spark workflow, with safety/idempotency guidance.
docs/Explore Algorithms/OpenAI/Quickstart - Multimodal OpenAI Prompter with Responses API.ipynb Adds a “next steps” section pointing users to the tool-calling workflow.
core/src/test/scala/com/microsoft/azure/synapse/ml/param/VerifyUntypedArrayParam.scala Expands test coverage for widened AnyJsonFormat encoding/decoding (null/Long/Float/JsValue).
core/src/main/scala/com/microsoft/azure/synapse/ml/param/UntypedArrayParam.scala Widens AnyJsonFormat to support JsValue, null, Long, and Float serialization, and JsNull reads.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/ToolTestFixtures.scala Introduces shared fixtures for tool JSON and representative Responses payloads.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtilsSuite.scala Adds unit tests for tool parsing/normalization, tool choice validation, and schema contracts.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolsLiveSuite.scala Adds credential-gated live smoke tests for real tool calls and continuations.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesToolsSuite.scala Adds comprehensive offline tests for serialization, per-row tool params, continuation ordering, and response parsing behavior.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesToolsIntegrationSuite.scala Adds local HTTP stub integration tests to verify executor behavior without external dependencies.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesSuite.scala Updates tests to use the internal V2 Responses schema.
cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala Adds Prompt-level tests ensuring tool/modern params forwarding and explicit Responses-only validation.
cognitive/src/test/python/synapsemltest/services/openai/test_OpenAIResponsesTools.py Adds Python tests for tool setter ergonomics, column helpers, and save/load behavior.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtils.scala Adds pure tool/toolChoice/input-items parsing + normalization + validation utilities for Responses tool payloads.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolPythonOverrides.scala Adds generated Python convenience methods for tool configuration and column helpers.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolParams.scala Adds shared tool params + Prompt opt-in structured outputs (toolCallsCol, responseStructCol).
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala Adds Spark column helpers for extracting tool calls and deterministic replay items from Responses structs.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAISchemas.scala Adds an internal V2 Responses schema to widen parsing while keeping public DTOs stable.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponsesModernParams.scala Adds modern Responses parameters and merging logic (including nested reasoning extras).
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala Implements tool payload support, continuation inputs, V2 schema parsing, tool call projection, and row-level validation.
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPrompt.scala Wires tool + modern Responses params through Prompt, adds Responses-only validation, and adds optional structured outputs.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (2)

cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala:139

  • The standalone spark expression is a no-op and can be removed (it doesn’t affect the test and can trip style checks for unused expressions).
    spark
    val prompt = new OpenAIPrompt()

cognitive/src/test/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPromptToolsSuite.scala:161

  • The standalone spark expression is a no-op and can be removed (it doesn’t affect the test and can trip style checks for unused expressions).
    spark
    val prompt = configuredPrompt
  • Files reviewed: 20/20 changed files
  • Comments generated: 2
  • Review effort level: Lite

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 13, 2026
## Summary
Fix the active review findings and failed PR validation for OpenAI Responses tool calling. Empty message arrays now skip correctly, continuation-only requests remain supported, OpenAIPrompt tests initialize Spark explicitly, the Databricks notebook avoids mutating a static Spark setting, and the complete PR patch applies and compiles on spark4.1.

## Prompting Intent
The engineer asked to fix every failed test and active review comment on PR microsoft#2626, rebase onto the latest master branch, preserve the new Responses and OpenAIPrompt tool-calling API, and rerun Azure Pipelines validation.

## Linked Sources
- Pull request and review comments: microsoft#2626
- Failed Azure Pipelines build 230906067: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230906067
- Requirements and reviewed design: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
Keep the release replay structural rather than branch-specific. Prompt-only tool parameters are layered through an internal mixin alias so master retains its wrapper model while spark4.1 retains its existing wrapper divergence without merge conflicts. Continuation items are assembled through the optional request map and production transforms inject an empty typed messages column only when needed, preserving the established prepareEntity shape across Scala 2.12 and 2.13. Tests use an explicit Spark-session assertion instead of a bare expression, and the notebook documents cluster-time speculation configuration because managed runtimes reject changing it at runtime.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Copilot AI review requested due to automatic review settings August 13, 2026 02:13
@ranadeepsingh
Rana Singh (ranadeepsingh) force-pushed the feat/openai-responses-tool-calling branch from 7bb0410 to 2953c2c Compare August 13, 2026 02:13
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala:161

  • prepareEntity calls .toSeq on the result of r.getAs[Seq[Row]](getMessagesCol). If messagesCol exists but the row value is null (common for continuation-only rows that still carry a nullable messages column), this will throw a NPE before the request is built. Treat null messages as an empty sequence so stored continuations can run without requiring non-null messages per row.
  override protected[openai] def prepareEntity: Row => Option[AbstractHttpEntity] = {
    r =>
      lazy val optionalParams: Map[String, Any] = getOptionalParams(r)
      val messages = r.getAs[scala.collection.Seq[Row]](getMessagesCol).toSeq
      Some(getStringEntity(messages, optionalParams))
  }
  • Files reviewed: 20/20 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 13, 2026
## Summary
Fix the active review findings and failed PR validation for OpenAI Responses tool calling. Empty message arrays now skip correctly, continuation-only requests remain supported, OpenAIPrompt tests initialize Spark explicitly, the Databricks notebook avoids mutating a static Spark setting, and the complete PR patch applies and compiles on spark4.1.

## Prompting Intent
The engineer asked to fix every failed test and active review comment on PR microsoft#2626, rebase onto the latest master branch, preserve the new Responses and OpenAIPrompt tool-calling API, and rerun Azure Pipelines validation.

## Linked Sources
- Pull request and review comments: microsoft#2626
- Failed Azure Pipelines build 230906067: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230906067
- Requirements and reviewed design: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
Keep the release replay structural rather than branch-specific. Prompt-only tool parameters are layered through an internal mixin alias so master retains its wrapper model while spark4.1 retains its existing wrapper divergence without merge conflicts. Continuation items are assembled through the optional request map and production transforms inject an empty typed messages column only when needed, preserving the established prepareEntity shape across Scala 2.12 and 2.13. Tests use an explicit Spark-session assertion instead of a bare expression, and the notebook documents cluster-time speculation configuration because managed runtimes reject changing it at runtime.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 13, 2026
## Summary
Add tool and function calling to OpenAIChatCompletion and the default Chat Completions path in OpenAIPrompt. Function definitions and named choices are converted to the Chat wire schema, assistant tool calls and tool result messages are preserved for continuations, and tool calls project to the same DataFrame contract as Responses. Update the live Responses bad-input assertion to match intentional empty-message skipping.

## Prompting Intent
The engineer asked to fix the failing tests on PR microsoft#2626 without unnecessary blank-line churn, rebase onto current master, and support tool calling through Chat Completions rather than rejecting it when OpenAIPrompt uses its default API type.

## Linked Sources
- Pull request: microsoft#2626
- Failed Azure Pipelines build 230948799: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230948799
- OpenAI Chat Completions API: https://developers.openai.com/api/reference/resources/chat
- OpenAI function calling guide: https://developers.openai.com/api/docs/guides/function-calling
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
Keep one Spark-facing tools and toolCallsCol contract while translating only at the service boundary: Responses uses flat function definitions and Chat Completions uses nested function objects. Preserve public response DTO arities through an internal Chat response schema, and require explicit assistant tool_calls plus role=tool messages for Chat continuations rather than executing tools automatically. Separate common tool parameters from Responses-only maxToolCalls so generated Chat APIs match the endpoint signature. Per-row validation shadows invalid message rows before HTTP execution, preserving DataFrame error isolation and at-least-once request semantics.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Copilot AI review requested due to automatic review settings August 13, 2026 05:50
@ranadeepsingh
Rana Singh (ranadeepsingh) force-pushed the feat/openai-responses-tool-calling branch from 2953c2c to dffdc0d Compare August 13, 2026 05:50
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@ranadeepsingh Rana Singh (ranadeepsingh) changed the title feat: add tool calling to OpenAI Responses and Prompt feat: add OpenAI tool calling for Responses, Chat, and Prompt Aug 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala:37

  • FunctionCallOutputStructType does not set containsNull = false, but toFunctionCallOutputs assumes every array element is a non-null Row and will throw if a null element is present. Mark the array as non-nullable and fail fast with a clear error if a null element still appears.
  val FunctionCallOutputStructType: ArrayType = ArrayType(StructType(Seq(
    StructField("call_id", StringType),
    StructField("output", StringType),
    StructField("status", StringType)
  )))
  • Files reviewed: 23/23 changed files
  • Comments generated: 1
  • Review effort level: Lite

Rana Singh (ranadeepsingh) pushed a commit to ranadeepsingh/SynapseML that referenced this pull request Aug 13, 2026
## Summary
Update the Python tool-calling tests for Chat Completions support and synchronize Python-side OpenAIPrompt parameters before API-dependent toolCallsColumn and replayItemsColumn helpers call the JVM.

## Prompting Intent
The engineer asked to fix all failing tests on PR microsoft#2626 while preserving dual Responses and Chat Completions tool calling, avoiding unnecessary whitespace changes, and rerunning Azure Pipelines validation.

## Linked Sources
- Pull request: microsoft#2626
- Failed Azure Pipelines build 230974693: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230974693
- Failing shard: PythonTests cognitive
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
The previous Python assertions treated tools and replay output as Responses-only, but tools are now shared with Chat Completions while maxToolCalls and replayItemsColumn remain Responses-only. Direct Python column-helper calls also bypassed PySpark's normal transform-time parameter transfer, so setApiType("responses") was not visible to the Java projection method. Reusing PySpark's standard parameter synchronization fixes both Chat and Responses projection selection without changing the public API or duplicating endpoint logic in Python.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Copilot AI review requested due to automatic review settings August 13, 2026 07:43
@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolColumns.scala:30

  • FunctionCallOutputStructType currently allows null array elements and marks call_id/output/status as nullable, but toFunctionCallOutputs later requires non-blank call_id and non-null output (and will NPE if an array element itself is null). Tightening the schema to containsNull=false and making call_id/output non-nullable helps catch invalid continuation rows earlier (schema/analysis) and keeps the contract consistent with ToolCallStructType.
  val FunctionCallOutputStructType: ArrayType = ArrayType(StructType(Seq(
    StructField("call_id", StringType),
    StructField("output", StringType),
    StructField("status", StringType)
  )))
  • Files reviewed: 23/23 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

SynapseML CI and others added 17 commits September 3, 2026 19:47
Fix the active review findings and failed PR validation for OpenAI Responses tool calling. Empty message arrays now skip correctly, continuation-only requests remain supported, OpenAIPrompt tests initialize Spark explicitly, the Databricks notebook avoids mutating a static Spark setting, and the complete PR patch applies and compiles on spark4.1.

The engineer asked to fix every failed test and active review comment on PR microsoft#2626, rebase onto the latest master branch, preserve the new Responses and OpenAIPrompt tool-calling API, and rerun Azure Pipelines validation.

- Pull request and review comments: microsoft#2626
- Failed Azure Pipelines build 230906067: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230906067
- Requirements and reviewed design: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

Keep the release replay structural rather than branch-specific. Prompt-only tool parameters are layered through an internal mixin alias so master retains its wrapper model while spark4.1 retains its existing wrapper divergence without merge conflicts. Continuation items are assembled through the optional request map and production transforms inject an empty typed messages column only when needed, preserving the established prepareEntity shape across Scala 2.12 and 2.13. Tests use an explicit Spark-session assertion instead of a bare expression, and the notebook documents cluster-time speculation configuration because managed runtimes reject changing it at runtime.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Add tool and function calling to OpenAIChatCompletion and the default Chat Completions path in OpenAIPrompt. Function definitions and named choices are converted to the Chat wire schema, assistant tool calls and tool result messages are preserved for continuations, and tool calls project to the same DataFrame contract as Responses. Update the live Responses bad-input assertion to match intentional empty-message skipping.

The engineer asked to fix the failing tests on PR microsoft#2626 without unnecessary blank-line churn, rebase onto current master, and support tool calling through Chat Completions rather than rejecting it when OpenAIPrompt uses its default API type.

- Pull request: microsoft#2626
- Failed Azure Pipelines build 230948799: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230948799
- OpenAI Chat Completions API: https://developers.openai.com/api/reference/resources/chat
- OpenAI function calling guide: https://developers.openai.com/api/docs/guides/function-calling
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

Keep one Spark-facing tools and toolCallsCol contract while translating only at the service boundary: Responses uses flat function definitions and Chat Completions uses nested function objects. Preserve public response DTO arities through an internal Chat response schema, and require explicit assistant tool_calls plus role=tool messages for Chat continuations rather than executing tools automatically. Separate common tool parameters from Responses-only maxToolCalls so generated Chat APIs match the endpoint signature. Per-row validation shadows invalid message rows before HTTP execution, preserving DataFrame error isolation and at-least-once request semantics.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Update the Python tool-calling tests for Chat Completions support and synchronize Python-side OpenAIPrompt parameters before API-dependent toolCallsColumn and replayItemsColumn helpers call the JVM.

## Prompting Intent
The engineer asked to fix all failing tests on PR microsoft#2626 while preserving dual Responses and Chat Completions tool calling, avoiding unnecessary whitespace changes, and rerunning Azure Pipelines validation.

## Linked Sources
- Pull request: microsoft#2626
- Failed Azure Pipelines build 230974693: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230974693
- Failing shard: PythonTests cognitive
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
The previous Python assertions treated tools and replay output as Responses-only, but tools are now shared with Chat Completions while maxToolCalls and replayItemsColumn remain Responses-only. Direct Python column-helper calls also bypassed PySpark's normal transform-time parameter transfer, so setApiType("responses") was not visible to the Java projection method. Reusing PySpark's standard parameter synchronization fixes both Chat and Responses projection selection without changing the public API or duplicating endpoint logic in Python.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Read Chat Completions messages as their actual Spark array type and skip both null and empty message arrays before HTTP execution.

## Prompting Intent
The engineer asked to fix all active PR comments and failing tests while keeping the OpenAI tool-calling changes surgical and avoiding unrelated whitespace churn.

## Linked Sources
- Pull request: microsoft#2626
- Review comment: microsoft#2626 (comment)
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
The messages column is an array of rows, so reading it as a single Row can throw at runtime. Matching the existing request encoder's scala.collection.Seq type preserves Scala 2.12 and 2.13 compatibility, while forall(_.isEmpty) treats both null and empty arrays as absent and prevents invalid rows from reaching the service.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Update the Azure OpenAI and AI Foundry Chat robustness suites to expect skipped empty message arrays to produce neither output nor an error.

## Prompting Intent
The engineer asked to fix every failing PR test and active review comment while preserving the intentional Chat Completions tool-calling behavior and avoiding unrelated formatting changes.

## Linked Sources
- Pull request: microsoft#2626
- Failed Azure Pipelines build 230996030: https://msdata.visualstudio.com/SynapseML/_build/results?buildId=230996030
- Review comment establishing empty-array skip behavior: microsoft#2626 (comment)
- Requirements and review context: Copilot session ae945b9b-3010-450e-a313-5ff439dad2a5
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
Both live suites used an empty messages array as a deliberately bad HTTP request and expected a service error. The reviewed implementation now treats empty arrays like missing input and skips them before HTTP execution, matching the Responses behavior and row-level validation contract. The tests therefore assert empty output and error columns for that row while preserving their existing checks for other malformed and null inputs.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Update the master-added offline schema assertions to use the widened internal Chat and Responses V2 schemas exposed by the tool-calling stages.

## Prompting Intent
The engineer asked to rebase PR microsoft#2626 onto the latest master, fix every resulting test failure, and avoid unnecessary blank-line or formatting changes.

## Linked Sources
- Pull request: microsoft#2626
- Failed Azure Pipelines build: https://msdata.visualstudio.com/A365/_build/results?buildId=231016727
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
The production stages intentionally parse widened internal V2 wire schemas so tool calls and modern Responses fields are available without changing public response case-class arities. Updating only the two exact-schema assertions preserves that compatibility design and keeps the rebase fix surgical.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Restore the legacy Chat and Responses responseDataType contracts while using widened V2 schemas only in the internal JSON parsers. Add regression coverage proving tool fields remain available in transformed DataFrames.

The engineer asked to rebase PR microsoft#2626 onto current master, fix every failed test and active comment, preserve compatibility, avoid unnecessary whitespace changes, and keep Azure plus Spark 4.1 validation green.

- Pull request: microsoft#2626
- Public-schema failure: https://msdata.visualstudio.com/A365/_build/results?buildId=231016727
- Spark 4.1 replay failure: https://msdata.visualstudio.com/A365/_build/results?buildId=231023446
- Azure Boards: no work item ID was provided for this GitHub contribution

The master test correctly guards responseDataType as a public compatibility surface. Returning the widened wire schemas there would expose internal parsing changes and break that contract. Overriding only getInternalOutputParser preserves the public DTO schemas while still parsing tool calls and modern Responses fields into V2 output structs. Reverting the master-only test edit also keeps the PR patch replayable on spark4.1, where that test file does not exist.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Add a credential-gated Azure OpenAI Chat Completions test that forces a named weather tool call, validates its structured arguments, replays the assistant call with a matching tool result, and verifies the model produces a grounded final answer.

## Prompting Intent
The engineer asked for meaningful end-to-end Chat Completions tool-calling coverage against the real Foundry/Azure OpenAI service, in addition to the existing deterministic mock and local HTTP tests, without unrelated whitespace changes.

## Linked Sources
- Pull request: microsoft#2626
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
A two-turn live test validates more than request acceptance: it proves the Azure service accepts SynapseML's named tool-choice payload, returns a usable call ID and JSON arguments, accepts the assistant/tool continuation message shape, and incorporates the tool result into final text. Keeping it in the existing credential-gated live suite avoids exposing secrets to offline tests and lets Azure Pipelines provide the authoritative service validation.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Make the credential-gated Chat Completions continuation test deterministic by framing tool results as authoritative, returning a self-describing weather payload, and explicitly asking the model to report the preceding tool value.

## Prompting Intent
The engineer asked for meaningful end-to-end Chat Completions tool-calling coverage against the real Foundry/Azure OpenAI service, including a complete assistant tool-call and tool-result continuation rather than mock-only coverage.

## Linked Sources
- Pull request: microsoft#2626
- Azure Boards: no work item ID was provided for this GitHub contribution

## Rationale
The initial live assertion exposed nondeterministic model behavior: Azure accepted the valid continuation payload, but GPT-5.1 sometimes declined to state current weather instead of volunteering the terse tool value. The follow-up user message asks for the exact value from the preceding tool message without embedding that value itself, so a passing assertion still proves the model received and used the structured tool result while avoiding reliance on discretionary phrasing.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Add a dedicated OpenAI tool-use tutorial covering Chat Completions and Responses API DataFrame workflows, then link it from the main OpenAI guide and website sidebar.

## Prompting Intent
The engineer asked for a new notebook like the existing OpenAI tutorial that demonstrates tool use, with meaningful end-to-end patterns for both supported OpenAI APIs and clear, intuitive Spark DataFrame usage.

## Linked Sources
- Pull request: microsoft#2626
- Main OpenAI tutorial: docs/Explore Algorithms/OpenAI/OpenAI.ipynb
- Responses tool-calling quickstart: docs/Explore Algorithms/OpenAI/Quickstart - OpenAI Responses Tool Calling.ipynb

## Rationale
Use a focused notebook instead of expanding the already broad OpenAI guide. A shared weather tool keeps the two API protocols directly comparable, while explicit argument validation, tool-result joins, continuation messages, materialization, and call-id idempotency show the production concerns unique to distributed Spark execution without hiding application-owned tool execution.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
## Summary
Use generated Docusaurus document routes for the OpenAI tool-use links so the website build resolves both the tutorial entry point and its deeper Responses quickstart.

## Prompting Intent
The engineer asked for the tool-use notebook to be published in PR microsoft#2626 with failing tests and checks fixed, while avoiding unrelated whitespace or notebook churn.

## Linked Sources
- Pull request: microsoft#2626
- Failed website run: https://github.com/microsoft/SynapseML/actions/runs/31747799767
- Azure validation build: https://msdata.visualstudio.com/A365/_build/results?buildId=231080451

## Rationale
Docusaurus serves converted notebooks as extensionless document routes. Removing the source `.ipynb` suffix and using a parent-relative sibling route fixes both broken links while retaining useful relative navigation in generated documentation and changing only the two failing link targets.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: ae945b9b-3010-450e-a313-5ff439dad2a5
Addresses the two suppressed Copilot review comments on this PR.

`FunctionCallOutputStructType` is declared as `ArrayType(StructType(...))`,
which defaults to `containsNull = true`, so a user-supplied outputs array may
legally contain null elements. `toFunctionCallOutputs` then called
`row.getAs[String]("call_id")` directly and threw a bare NullPointerException
instead of the actionable IllegalArgumentException the surrounding validation
already produces for blank call_id, null output, and duplicate call_id.

Guard each element with `Option(row)` and raise the same
`function_call_output <index>: ...` style error used by the neighbouring
checks. The guard is enforced at runtime rather than by tightening the public
schema to `containsNull = false`, because that schema is part of the
user-facing contract and Spark does not reliably enforce declared nullability
on incoming data, so a runtime check is the stronger fix.

Verified by reverting only the source change: the new assertion fails with
"Expected exception java.lang.IllegalArgumentException to be thrown, but
java.lang.NullPointerException was thrown", and passes with the fix.

Co-authored-by: Copilot <[email protected]>
GitHub-PR: microsoft#2626

## Summary
Fix Responses continuation-only execution after rebasing, validate strict
function schemas recursively, prevent public output-column collisions, and
document Azure's strict-schema parallel-call restriction. Extract Responses
message encoding into a focused helper to keep the service implementation
reviewable.

## Prompting Intent
The engineer asked to rebase PR microsoft#2626, verify current Chat Completions and
Responses tool calling for gpt-5-mini and gpt-5.1, correct discrepancies, and
prepare an explanatory PDF with properly formatted and highlighted examples.
All work had to remain isolated in the existing PR worktree.

## Linked Sources
- Pull request: microsoft#2626
- OpenAI function calling guide: https://developers.openai.com/api/docs/guides/function-calling
- OpenAI conversation state guide: https://developers.openai.com/api/docs/guides/conversation-state
- OpenAI reasoning guide: https://developers.openai.com/api/docs/guides/reasoning
- OpenAI gpt-5-mini model page: https://developers.openai.com/api/docs/models/gpt-5-mini
- OpenAI gpt-5.1 model page: https://developers.openai.com/api/docs/models/gpt-5.1
- Azure OpenAI function calling: https://learn.microsoft.com/azure/ai-foundry/openai/how-to/function-calling
- Azure structured outputs: https://learn.microsoft.com/azure/ai-foundry/openai/how-to/structured-outputs
- PR review threads: microsoft#2626 (comment)

## Rationale
Preserve the PR's API-neutral Spark surface while adapting function definitions
to each endpoint's wire format. Fail locally for provider-independent strict
JSON Schema violations, but warn rather than reject provider-specific Azure
strict-plus-parallel combinations because custom proxies and endpoint
capabilities can differ. Keep function execution application-controlled and
correlate every result by call ID, not function name.

Co-authored-by: Copilot <[email protected]>
GitHub-PR: microsoft#2626

## Summary
Declare Responses function-call output arrays and their required fields as
non-nullable, and make Chat Completions reject the Responses-only replay helper
with a clear unsupported-operation error.

## Prompting Intent
The engineer asked to rebase PR microsoft#2626, verify modern Chat Completions and
Responses tool calling for gpt-5-mini and gpt-5.1, correct all discrepancies,
and leave the pull request merge-ready with an explanatory PDF. This follow-up
addresses the two remaining unresolved review findings on the rebased head.

## Linked Sources
- Pull request: microsoft#2626
- Function-call output schema review: microsoft#2626 (comment)
- Chat replay helper review: microsoft#2626 (comment)
- OpenAI function calling guide: https://developers.openai.com/api/docs/guides/function-calling

## Rationale
The continuation schema is new in this unmerged change, so its declared Spark
type should accurately state that array elements, call IDs, and outputs are
required. Runtime validation remains in place because Spark nullability metadata
is not an enforcement boundary and inferred schemas may be more permissive.
Overriding the inherited replay helper on Chat preserves the shared tool-call
projection surface while preventing a Responses-shaped expression from failing
later during Spark analysis; generated Python continues to omit that helper.

Co-authored-by: Copilot <[email protected]>
GitHub-PR: microsoft#2626

## Summary
Treat empty Chat Completions message arrays like null input rows during
transform validation so they are skipped without an output, error, or HTTP
request. Add a local HTTP executor regression for both null and empty arrays.

## Prompting Intent
The engineer asked to make PR microsoft#2626 merge-ready after rebasing and to correct
any discrepancies in SynapseML's modern OpenAI tool-calling behavior. A
final-head review found that the public bad-input contract expected empty
message arrays to be silently skipped while structured validation emitted an
error before the inherited skip logic ran.

## Linked Sources
- Pull request: microsoft#2626
- Review finding: microsoft#2626 (comment)
- OpenAI Chat Completions API: https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create

## Rationale
The transformer already treats null and empty arrays as rows with no request to
send. Applying structural validation only to present messages makes that
behavior consistent across validation, shouldSkip, and the existing robustness
test. Non-empty malformed messages still produce row-isolated errors, while
the executor-level test proves skipped rows never reach the HTTP server.

Co-authored-by: Copilot <[email protected]>
## Summary
Replay the minimal missing OpenAI multimodal prerequisite chain on spark4.1
and keep the Responses helper block mergeable with that branch's Scala 2.13
collection adaptations.

## Prompting Intent
The engineer asked to rebase GitHub PR microsoft#2626, verify current Chat Completions
and Responses tool calling, correct discrepancies, and make the PR
merge-ready from an isolated worktree.

## Linked Sources
- Pull request: microsoft#2626
- OpenAI function calling guide: https://platform.openai.com/docs/guides/function-calling
- OpenAI Responses API reference: https://platform.openai.com/docs/api-reference/responses
- OpenAI Chat Completions API reference: https://platform.openai.com/docs/api-reference/chat/create

## Rationale
Use the repository's existing release-compatibility prerequisite mechanism
instead of changing pipeline logic or adding port-specific behavior to
master. Alternate-index replay reduced the prerequisite set from 21
candidates to the 15 commits actually required. Relocating private helper
methods is behavior-neutral and lets three-way replay preserve spark4.1's
Scala 2.13 Seq conversions; normalizing null messages in the shared entity
builder preserves continuation-only requests.

Co-authored-by: Copilot <[email protected]>
## Summary
Validate Chat message content before recursively encoding the full row so
short rows produce stable structural errors, and align the multimodal suites
with the documented null/empty-message skip contract.

## Prompting Intent
The engineer asked to make GitHub PR microsoft#2626 merge-ready after rebasing it,
verify modern OpenAI tool calling, correct discrepancies, and preserve
cross-runtime SynapseML behavior.

## Linked Sources
- Pull request: microsoft#2626
- Exact-head Azure build: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=232941908
- Failed OpenAI job: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=232941908&view=logs&jobId=30886130-6c10-51cc-2b7c-fdfe28a165e1

## Rationale
The recursive serializer accessed schema fields before the dedicated
role/content validators, leaking ArrayIndexOutOfBoundsException for malformed
Rows. Computing validated content first preserves valid payloads while
restoring the intended IllegalArgumentException contract. Empty message arrays
already skip HTTP execution by design, so the older multimodal assertions now
match the public no-request/no-output/no-error behavior instead of
dereferencing a deliberately absent error.

Co-authored-by: Copilot <[email protected]>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A docs cross-link in docs/Explore Algorithms/OpenAI/OpenAI.ipynb likely resolves to a nested (incorrect) path under trailing-slash routing and should be updated to a parent-relative link to avoid a 404.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1 Low severity

New issues introduced by this change (1)
Severity Finding
Low severity docs/​Explore Algorithms/​OpenAI/​OpenAI.ipynb — The relative link to OpenAI_ToolUse is likely incorrect under the docs site’s `trailingSlash:…

Comment thread docs/Explore Algorithms/OpenAI/OpenAI.ipynb
## Summary

Use a parent-relative documentation link so the OpenAI overview reaches the sibling tool-use page under trailing-slash routing.

## Prompting Intent

The engineer asked to rebase and validate every open pull request before authorizing Azure Pipelines. Copilot review of the rebased OpenAI PR found that the new cross-link would resolve beneath the current page and return a 404, so this commit fixes the link without changing notebook execution.

## Linked Sources

- Copilot finding: microsoft#2626 (comment)
- Pull request: microsoft#2626

## Rationale

The generated docs use trailing-slash page URLs, making ../OpenAI_ToolUse the correct sibling route. This matches the parent-relative links already used by the tool-use notebook.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 476d113f-dd35-40c6-bc79-005dcccd7b79

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The change set substantially expands OpenAI request/response/continuation behavior across multiple transformers and schemas, so it needs final human review despite only minor localized feedback.

Review tier: Lite
Findings: None

Issues resolved since last review (1)
Severity Finding
Low severity docs/​Explore Algorithms/​OpenAI/​OpenAI.ipynb — The relative link to OpenAI_ToolUse is likely incorrect under the docs site’s `trailingSlash:… View resolved comment
Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala:182

  • encodeChatMessagesToMap is now dead code (no call sites in this file). Keeping an unused private helper makes the message-encoding path harder to follow and can hide future regressions when refactoring.

Remove the unused method (or wire it into getStringEntity... if it’s meant to be the canonical encoding entry point).

## Summary

Remove an unused Chat message encoding helper and restore the OpenAI tool-use link to the path required by the generated directory-index route.

## Prompting Intent

The engineer asked to rebase and validate all open pull requests before authorizing Azure Pipelines. The final Copilot review surfaced an unused helper, while the GitHub website build proved Copilot's parent-relative link suggestion resolved outside the OpenAI directory; this commit clears both blockers.

## Linked Sources

- Copilot review: microsoft#2626 (review)
- Website failure: https://github.com/microsoft/SynapseML/actions/runs/33833625636/job/100901574523
- Link discussion: microsoft#2626 (comment)

## Rationale

Removing the unreachable private helper simplifies the active encoding path without changing behavior. The OpenAI overview is emitted as the directory index, so ./OpenAI_ToolUse correctly targets its nested sibling page; the failed Docusaurus resolution provides direct evidence that ../ escapes one level too far.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 476d113f-dd35-40c6-bc79-005dcccd7b79

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It introduces broad cross-endpoint API, schema, and serialization changes with significant surface area that warrants final human review despite strong test coverage.

Review tier: Lite
Findings: 1 Low severity

New issues introduced by this change (1)
Severity Finding
Low severity website/​sidebars.js — A new notebook Quickstart - OpenAI Responses Tool Calling is added/linked elsewhere in this PR,…

Comment thread website/sidebars.js
## Summary

Add the Responses tool-calling quickstart to the OpenAI documentation sidebar so the new notebook is discoverable from site navigation.

## Prompting Intent

The engineer asked to rebase and fully validate every open pull request before authorizing Azure Pipelines. Copilot's final exact-head review found that the new quickstart was linked from content but omitted from the OpenAI sidebar, so this commit closes that discoverability gap.

## Linked Sources

- Copilot finding: microsoft#2626 (comment)
- Pull request: microsoft#2626

## Rationale

A direct sidebar entry follows the existing OpenAI quickstart convention and makes the shipped notebook reachable without requiring readers to discover an inline cross-link first.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 476d113f-dd35-40c6-bc79-005dcccd7b79

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It introduces a large new OpenAI tool-calling surface area (new schemas/params, request assembly, and generated Python overrides) plus core JSON param widening, which warrants careful human review beyond this automated pass.

Review tier: Lite
Findings: None

Issues resolved since last review (1)
Severity Finding
Low severity website/​sidebars.js — A new notebook Quickstart - OpenAI Responses Tool Calling is added/linked elsewhere in this PR,… View resolved comment
Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

core/src/main/scala/com/microsoft/azure/synapse/ml/param/UntypedArrayParam.scala:26

  • AnyJsonFormat serializes Float via BigDecimal(v.toString), which will throw a low-signal NumberFormatException for NaN/Infinity (since those strings aren’t valid decimals). If those values are intended to be rejected, it’s better to fail with an explicit, actionable error message.

## Summary

Reject NaN and infinite Float values with an explicit IllegalArgumentException when serializing untyped array parameters, and cover all three non-finite forms with regression tests.

## Prompting Intent

The engineer asked to make every rebased pull request merge-ready before authorizing credential-bearing Azure Pipelines. The latest exact-head review identified that non-finite Float values leaked a low-signal decimal parser exception, so this commit resolves that finding.

## Linked Sources

- Pull request: microsoft#2626

## Rationale

Validate Float finiteness before constructing BigDecimal so callers receive a stable, actionable serialization error instead of an implementation-specific NumberFormatException. Finite Float rendering remains unchanged.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 476d113f-dd35-40c6-bc79-005dcccd7b79

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

A couple of concrete issues in newly introduced tool serialization/parameter validation (notably non-finite number handling in OpenAIToolUtils.toJsValue and a misleading setTools error message) should be addressed to ensure robust, actionable failures for users.

Review tier: Lite
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolUtils.scala:57

  • OpenAIToolUtils.toJsValue serializes Double/Float via BigDecimal(value.toString). For non-finite values (NaN/Infinity), this will throw a NumberFormatException, which is both non-obvious and inconsistent with the clearer AnyJsonFormat Float handling. Explicitly reject non-finite Double/Float with an IllegalArgumentException so callers get a stable, actionable error.
    cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIToolParams.scala:69
  • The guard in setTools only checks for a non-blank string, but the error message says the JSON array must be non-empty. This is misleading ("[]" is valid) and makes debugging blank/null inputs harder. Adjust the message to match the actual validation (non-blank JSON array string).

## Summary

Reject non-finite floating-point values in OpenAI tool payloads with explicit errors and align the string-based setTools error message with its actual non-blank input requirement.

## Prompting Intent

The engineer asked to make every rebased pull request merge-ready before authorizing credential-bearing Azure Pipelines. The latest exact-head Copilot review identified two suppressed validation findings, so this commit resolves both with regression coverage.

## Linked Sources

- Pull request: microsoft#2626

## Rationale

Validate Double and Float finiteness before constructing JSON numbers so callers receive stable IllegalArgumentExceptions instead of parser failures. Keep an empty JSON array valid and describe only the non-blank constraint that setTools actually enforces.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 476d113f-dd35-40c6-bc79-005dcccd7b79

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The change set spans multiple public Spark transformers (Scala + generated Python surfaces) plus schema/serialization and extensive new test infrastructure, warranting a final human compatibility review.

Review tier: Lite
Findings: None

@ranadeepsingh

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants