diff --git a/README.md b/README.md index cd7b245..2c91e63 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,6 @@
- - [![Discord](https://badgen.net/discord/members/fxjRrbMH3Q/?color=8978cc&icon=discord)](https://discord.com/invite/fxjRrbMH3Q) [![Twitter](https://badgen.net/badge/b/behavioralsignals/icon?icon=twitter&label&color=black)](https://x.com/behaviorsignals) [![readme.io](https://badgen.net/badge/readme.io/Documentation/?color=black)](https://behavioralsignals.readme.io/) @@ -25,106 +23,85 @@
-Python SDK for the Behavioral Signals API. Behavioral Signals builds AI solutions that understand human behavior through voice and detect deepfake content in audio. -Our API enables developers to integrate behavioral analysis into their applications, both in batch and streaming modes. -See the [API documentation](https://behavioralsignals.readme.io/) for details. It is also available as [llms.txt](https://behavioralsignals.readme.io/llms.txt) for AI coding assistants. - - -## Table of Contents -* [Behavioral Signals API Python SDK](#behavioral-signals-api-python-sdk) - * [Features](#features) - * [Requirements](#requirements) - * [API Key Setup](#api-key-setup) - * [SDK Installation](#sdk-installation) - * [SDK Example Usage](#sdk-example-usage) - * [Behavioral API Batch Mode](#behavioral-api-batch-mode) - * [Behavioral API Streaming Mode](#behavioral-api-streaming-mode) - * [Deepfakes API Batch Mode](#deepfakes-api-batch-mode) - * [Deepfakes API Streaming Mode](#deepfakes-api-streaming-mode) - * [Available Methods](#available-methods) - * [Error Handling](#error-handling) - * [Use with AI assistants (MCP)](#use-with-ai-assistants-mcp) - * [Running the MCP server manually](#running-the-mcp-server-manually) +Official Python SDK for the [Behavioral Signals API](https://behavioralsignals.readme.io/). -## Features +Analyze human behavior and detect deepfake speech using batch and real-time audio APIs. Experimental video deepfake detection is also available in batch mode. -- **Behavioral Analysis API** : Analyze human behavior in both batch (offline) and streaming (online) modes. +[Python SDK Documentation](https://behavioralsignals.readme.io/docs/behavioral-signals-python-sdk) · +[Examples](examples/) · +[PyPI](https://pypi.org/project/behavioralsignals/) · +[Contributing](CONTRIBUTING.md) -- **Deepfake Detection API**: Detect synthetic or manipulated speech using advanced deepfake detection models. - - Supports batch (offline) and streaming (online) modes - - Compatible with a wide range of spoken languages +## Features -- **Core Speech Attributes (Batch Only)**: Extract foundational conversational metadata from both APIs: - - Automatic Speech Recognition (ASR) - - Speaker Diarization - - Language Identification +- **Behavioral Analysis** — analyze human behavior from speech in batch and real-time streaming modes +- **Deepfake Detection** — detect synthetic or manipulated speech in batch and real-time streaming modes +- **Video Deepfake Detection (Experimental, Batch Only)** — analyze both the video frames and audio track of supported video files +- **Core Speech Attributes (Batch Only)** — automatic speech recognition (ASR), speaker diarization, and language identification +- **Embeddings** — retrieve speaker and behavioral embeddings from the Behavioral API, or speaker and deepfake embeddings from the Deepfakes API +- **S3 Input** — submit audio and video using S3 presigned URLs +- **MCP Server** — use Behavioral Signals from MCP-compatible AI assistants -- **MCP server**: Let AI assistants such as Claude and Cursor analyze files for you. See [Use with AI assistants (MCP)](#use-with-ai-assistants-mcp). +## Quickstart -## Requirements +### Install -* `Python3.10+`, -* `ffmpeg`, -* Python dependencies as specified in `pyproject.toml` +```bash +pip install behavioralsignals +``` +Requires Python 3.10 or later. -## API Key Setup +### Configure credentials -To use the Behavioral Signals API, you need to create an account and obtain an API key from the [Behavioral Signals portal](https://portal.behavioralsignals.com/). +Create an account and API key in the [Behavioral Signals portal](https://portal.behavioralsignals.com/). -You can pass your client ID (CID) and API key to `Client(YOUR_CID, YOUR_API_KEY)`, or set them as environment variables and call `Client()`: +Set your client ID (CID) and API key as environment variables: ```bash -export BEHAVIORALSIGNALS_CID=your_cid -export BEHAVIORALSIGNALS_API_KEY=your_api_key +export BEHAVIORALSIGNALS_CID="your_cid" +export BEHAVIORALSIGNALS_API_KEY="your_api_key" ``` -## SDK Installation +The SDK will pick them up automatically: -```bash -pip install behavioralsignals -``` - -## SDK Example Usage - -After obtaining your API key, you can use the SDK to interact with the Behavioral Signals APIs. -We currently provide two main APIs: +```python +from behavioralsignals import Client -* the **Behavioral API** for analyzing human behavior through voice, and -* the **Deepfakes API** for detecting deepfake audio content in human speech. +client = Client() +``` -Both APIs support batch and streaming modes, allowing you to send audio files or streams for analysis and receive results after processing and in real-time, respectively. -You can also find more detailed examples for both [batch](https://github.com/BehavioralSignalTechnologies/behavioralsignals-python/blob/main/examples/batch/README.md) and [streaming](https://github.com/BehavioralSignalTechnologies/behavioralsignals-python/blob/main/examples/streaming/README.md) in the `examples/` directory. +You can also pass the credentials directly: -### Behavioral API Batch Mode +```python +client = Client("your_cid", "your_api_key") +``` -In batch mode, you can send audio files to the Behavioral Signals API for analysis. The API will return a unique process ID (PID) that you can use to retrieve the results later. -`wait_for_result` checks the process status until processing is complete, then returns the results (available since version 0.6.0). -Pass `timeout` in seconds to stop waiting after that long; it raises `TimeoutError` if processing has not finished. +### Analyze audio ```python from behavioralsignals import Client +from behavioralsignals.utils import print_results -client = Client(YOUR_CID, YOUR_API_KEY) +client = Client() -response = client.behavioral.upload_audio(file_path="audio.wav") -output = client.behavioral.wait_for_result(pid=response.pid, timeout=600) +process = client.behavioral.upload_audio(file_path="audio.wav") +result = client.behavioral.wait_for_result( + pid=process.pid, + timeout=600, +) -for item in output.results or []: - top = item.prediction[0] - label = item.finalLabel or top.score # continuous tasks (e.g. intensity) have no label - if not label: - continue # the features row carries embeddings, not a result - confidence = f" ({float(top.posterior):.1%})" if top.posterior else "" - print(f"{item.st} {item.et} {item.task} {label}{confidence}") +print_results(result.results) ``` -Each result row has the start and end time in seconds, the task, and the top label with its probability. +`upload_audio()` returns a process with a unique process ID (`pid`). `wait_for_result()` polls until processing completes and returns the analysis result. + +`print_results()` prints one line per result: the start and end time in seconds, the task, and the top label with its probability. Continuous tasks such as `intensity` have no label, only a score, and `asr` and `diarization` have a label without a probability. For a 10-second clip of one speaker, the output starts like this: ``` -0.487 3.001 asr The birch canoe slid on the smooth plank. +0.487 3.001 asr The birch canoe slid on the smooth plank. 0.487 3.001 diarization SPEAKER_00 0.487 3.001 language en (98.9%) 0.487 3.001 gender female (99.7%) @@ -134,191 +111,223 @@ For a 10-second clip of one speaker, the output starts like this: 0.487 3.001 intensity 0.0873 ``` -Each row also has `prediction`, the list of all labels with their probabilities. Use `output.model_dump()` to get the results as a dictionary. +Each row also has `prediction`, the list of all labels with their probabilities. Use `result.model_dump()` to get the results as a dictionary. -Setting `embeddings=True` during audio upload will include speaker and behavioral embeddings in the output (see [documentation](https://behavioralsignals.readme.io/docs/embeddings#/)): +To include speaker and behavioral embeddings: ```python -response = client.behavioral.upload_audio(file_path="audio.wav", embeddings=True) -output = client.behavioral.wait_for_result(pid=response.pid, timeout=600) +process = client.behavioral.upload_audio( + file_path="audio.wav", + embeddings=True, +) ``` -Each HTTP request also has its own time limit: 10 seconds to connect, then up to 60 seconds of waiting for the server (300 seconds for uploads). -You can change these when creating the client, or pass `None` for no limit: +## Deepfake Detection + +### Audio + +Batch deepfake detection follows the same workflow as Behavioral Analysis: ```python -client = Client(YOUR_CID, YOUR_API_KEY, timeout=(10, 120), upload_timeout=(10, 900)) -``` +from behavioralsignals import Client +from behavioralsignals.utils import print_results + +client = Client() -These limits apply to single requests. The `timeout` of `wait_for_result` limits the whole wait. -If an upload times out, the server may still have received the file and started a process, so check `list_processes()` (or `list_video_processes()` for videos) before uploading again. +process = client.deepfakes.upload_audio(file_path="audio.wav") +result = client.deepfakes.wait_for_result( + pid=process.pid, + timeout=600, +) -### Behavioral API Streaming Mode +print_results(result.results) +``` -In streaming mode, you can send audio data in real-time to the Behavioral Signals API. The API will return results as they are processed. +To include speaker and deepfake embeddings: ```python -from behavioralsignals import Client, StreamingOptions -from behavioralsignals.utils import make_audio_stream - -client = Client(YOUR_CID, YOUR_API_KEY) -audio_stream, sample_rate = make_audio_stream("audio.wav", chunk_size=0.25) -options = StreamingOptions(sample_rate=sample_rate, encoding="LINEAR_PCM") - -for result in client.behavioral.stream_audio(audio_stream=audio_stream, options=options): - for item in result.results or []: - top = item.prediction[0] - label = item.finalLabel or top.score # continuous tasks (e.g. intensity) have no label - if not label: - continue # the features row carries embeddings, not a result - confidence = f" ({float(top.posterior):.1%})" if top.posterior else "" - print(f"{item.st} {item.et} {item.task} {label}{confidence}") +process = client.deepfakes.upload_audio( + file_path="audio.wav", + embeddings=True, +) ``` -### Deepfakes API Batch Mode +### Experimental Generator Detection -A similar example for the Deepfakes API in batch mode allows you to send audio files for deepfake detection: +Generator detection is an experimental batch feature that attempts to identify the model used to generate deepfake audio. ```python -from behavioralsignals import Client +process = client.deepfakes.upload_audio( + file_path="audio.wav", + enable_generator_detection=True, +) +``` -client = Client(YOUR_CID, YOUR_API_KEY) +See the [generator detection documentation](https://behavioralsignals.readme.io/docs/generator-detection) for details and supported generators. -response = client.deepfakes.upload_audio(file_path="audio.wav") -output = client.deepfakes.wait_for_result(pid=response.pid, timeout=600) +### Experimental Video Deepfake Detection -for item in output.results or []: - top = item.prediction[0] - label = item.finalLabel or top.score # some tasks have no label, only a score - if not label: - continue # the features row carries embeddings, not a result - confidence = f" ({float(top.posterior):.1%})" if top.posterior else "" - print(f"{item.st} {item.et} {item.task} {label}{confidence}") -``` - -Setting `embeddings=True` during audio upload will include speaker and deepfake embeddings in the output (see [documentation](https://behavioralsignals.readme.io/docs/embeddings-1#/)): +Video deepfake detection is available in **batch mode only** and is currently experimental. ```python -response = client.deepfakes.upload_audio(file_path="audio.wav", embeddings=True) -output = client.deepfakes.wait_for_result(pid=response.pid, timeout=600) +process = client.deepfakes.upload_video(file_path="video.mp4") +result = client.deepfakes.wait_for_video_result( + pid=process.pid, + timeout=600, +) ``` +Unlike an audio result, a video result contains two separate lists: -#### 🔬 Experimental: Deepfake Generator Prediction (Batch Only) - -An experimental option is now available that attempts to predict the generator model used to produce a deepfake. -When enabled, the returned results will contain an additional field - only for audios with detected deepfake content - indicating the predicted generator model along with a confidence score. +- `audio_results` — deepfake detection results for the video's audio track +- `video_results` — deepfake detection results for the video frames -You can activate this feature by passing `enable_generator_detection=True` during audio upload: +For example: ```python -from behavioralsignals import Client +print_results(result.video_results) +``` -client = Client(YOUR_CID, YOUR_API_KEY) +You can also submit a video using an S3 presigned URL with `upload_s3_presigned_video_url()` and inspect video processes using `list_video_processes()` and `get_video_process()`. -response = client.deepfakes.upload_audio(file_path="audio.wav", enable_generator_detection=True) -output = client.deepfakes.wait_for_result(pid=response.pid, timeout=600) -``` +See the [video deepfake documentation](https://behavioralsignals.readme.io/docs/submit-a-file-for-processing) for supported formats, limits, and the current experimental status. -See more in our [API documentation](https://behavioralsignals.readme.io/docs/generator-detection#/). +## Streaming -#### 🎬 Video Deepfake Detection (Batch Only) +Both the Behavioral and Deepfakes APIs support real-time **audio** streaming over gRPC. -In addition to audio, the Deepfakes API can detect deepfakes in video files. You upload a video the same way you upload audio, and the API analyzes both the audio track and the video frames. +To stream an audio file: ```python -from behavioralsignals import Client +from behavioralsignals import Client, StreamingOptions +from behavioralsignals.utils import make_audio_stream, print_results -client = Client(YOUR_CID, YOUR_API_KEY) +client = Client() -response = client.deepfakes.upload_video(file_path="video.mp4") -output = client.deepfakes.wait_for_video_result(pid=response.pid, timeout=600) -``` +audio_stream, sample_rate = make_audio_stream( + "audio.wav", + chunk_size=0.25, +) -Unlike `wait_for_result`, the video result response returns two separate lists — `audio_results` (deepfake detection on the audio track) and `video_results` (deepfake detection on the video frames): +options = StreamingOptions( + sample_rate=sample_rate, + encoding="LINEAR_PCM", +) -```python -for item in output.video_results or []: - top = item.prediction[0] - label = item.finalLabel or top.score # some tasks have no label, only a score - if not label: - continue # the features row carries embeddings, not a result - confidence = f" ({float(top.posterior):.1%})" if top.posterior else "" - print(f"{item.st} {item.et} {item.task} {label}{confidence}") +for result in client.behavioral.stream_audio( + audio_stream=audio_stream, + options=options, +): + print_results(result.results) ``` -You can also submit a video via an S3 presigned URL with `client.deepfakes.upload_s3_presigned_video_url(url=...)`, and list/inspect video processes with `client.deepfakes.list_video_processes()` and `client.deepfakes.get_video_process(pid=...)`. The `embeddings` and `enable_generator_detection` options are supported and apply to the audio-track results. Video deepfake detection is currently available in batch mode only. +`chunk_size` is specified in **seconds**, so `0.25` corresponds to 250 ms. -### Deepfakes API Streaming Mode +`stream_audio()` can also accept your own `Iterator[bytes]`, for example from a microphone or live call. -A similar streaming example for the Deepfakes API allows you to send audio data in real-time for speech deepfake detection: +For real-time deepfake detection, use the same interface: ```python -from behavioralsignals import Client, StreamingOptions -from behavioralsignals.utils import make_audio_stream - -client = Client(YOUR_CID, YOUR_API_KEY) -audio_stream, sample_rate = make_audio_stream("audio.wav", chunk_size=0.25) -options = StreamingOptions(sample_rate=sample_rate, encoding="LINEAR_PCM") - -for result in client.deepfakes.stream_audio(audio_stream=audio_stream, options=options): - for item in result.results or []: - top = item.prediction[0] - label = item.finalLabel or top.score # some tasks have no label, only a score - if not label: - continue # the features row carries embeddings, not a result - confidence = f" ({float(top.posterior):.1%})" if top.posterior else "" - print(f"{item.st} {item.et} {item.task} {label}{confidence}") +for result in client.deepfakes.stream_audio( + audio_stream=audio_stream, + options=options, +): + print_results(result.results) ``` +See the [streaming examples](examples/streaming/) and [streaming documentation](https://behavioralsignals.readme.io/docs/streaming-using-python-sdk) for more. + ## Available Methods -`client.behavioral` and `client.deepfakes` have the same methods for audio. Video methods are only on `client.deepfakes`. +`client.behavioral` and `client.deepfakes` expose the same methods for audio. Video methods are available only on `client.deepfakes`. | Method | What it does | |---|---| -| `upload_audio(file_path, ...)` | Uploads an audio file and returns the process, with its `pid` | -| `upload_s3_presigned_url(url, ...)` | Same as `upload_audio`, for audio at an S3 presigned URL | +| `upload_audio(file_path, ...)` | Uploads an audio file and returns the process, including its `pid` | +| `upload_s3_presigned_url(url, ...)` | Submits audio using an S3 presigned URL | | `wait_for_result(pid, timeout=None)` | Waits for a process to finish and returns its results | | `get_result(pid)` | Returns the results of a finished process | -| `get_process(pid)` | Returns a process and its status | -| `list_processes(page=0, page_size=1000, sort="asc", start_date=None, end_date=None)` | Lists your processes | -| `stream_audio(audio_stream, options)` | Sends audio as a stream and yields results as they arrive | +| `get_process(pid)` | Returns a process and its current status | +| `list_processes(...)` | Lists audio processes | +| `stream_audio(audio_stream, options)` | Streams audio and yields results as they arrive | | `upload_video(file_path, ...)` | Deepfakes only: uploads a video file | -| `upload_s3_presigned_video_url(url, ...)` | Deepfakes only: same as `upload_video`, for a video at an S3 presigned URL | +| `upload_s3_presigned_video_url(url, ...)` | Deepfakes only: submits video using an S3 presigned URL | | `wait_for_video_result(pid, timeout=None)` | Deepfakes only: waits for a video process to finish and returns its results | | `get_video_result(pid)` | Deepfakes only: returns the results of a finished video process | | `get_video_process(pid)` | Deepfakes only: returns a video process and its status | -| `list_video_processes(...)` | Deepfakes only: lists your video processes | +| `list_video_processes(...)` | Deepfakes only: lists video processes | + +`behavioralsignals.utils` also has two helpers: `make_audio_stream(file_path, chunk_size=0.25)` turns an audio file into chunks for `stream_audio()`, and `print_results(items)` prints result rows as shown in [Analyze audio](#analyze-audio). + +For detailed parameters and result schemas, see the [full API documentation](https://behavioralsignals.readme.io/). + +## Timeouts and Error Handling -## Error Handling +### Batch API -When the API returns an error, the SDK raises `BehavioralSignalsError`. Its `status_code` is the HTTP status code. -`wait_for_result` also raises `TimeoutError` if processing is not done within `timeout` seconds, and `RuntimeError` if the process failed (for example, not enough credits). +Batch HTTP API errors raise `BehavioralSignalsError`. Its `status_code` contains the HTTP status code. ```python from behavioralsignals import BehavioralSignalsError, Client -client = Client(YOUR_CID, YOUR_API_KEY) +client = Client() try: - output = client.behavioral.get_result(pid=12345) + result = client.behavioral.get_result(pid=12345) except BehavioralSignalsError as error: print(error.status_code, error) ``` -`BehavioralSignalsError` is a subclass of `Exception`, so code that catches `Exception` keeps working. +`wait_for_result()` and `wait_for_video_result()` also raise: + +- `TimeoutError` if processing has not finished within the supplied `timeout` +- `RuntimeError` if the process finishes unsuccessfully, for example because of insufficient credits + Network problems raise the usual `requests` exceptions, such as `requests.ConnectionError`. -## Use with AI assistants (MCP) +Each HTTP request also has its own timeout. By default, the SDK allows 10 seconds to connect and up to 60 seconds waiting for the server, or 300 seconds for uploads. + +You can customize these limits when creating the client: + +```python +client = Client( + timeout=(10, 120), + upload_timeout=(10, 900), +) +``` + +These request timeouts are separate from the overall `timeout` passed to `wait_for_result()`. + +> If an upload request times out, the server may still have received the file and started processing it. Check `list_processes()` — or `list_video_processes()` for videos — before uploading the same file again. + +### Streaming API + +Streaming uses gRPC rather than the batch HTTP API. Streaming failures can therefore raise `grpc.RpcError` instead of `BehavioralSignalsError`. + +```python +import grpc + +try: + for result in client.behavioral.stream_audio( + audio_stream=audio_stream, + options=options, + ): + ... # handle results +except grpc.RpcError as error: + print(error.code(), error.details()) +``` + +## Use with AI Assistants (MCP) + +The SDK includes an [MCP](https://modelcontextprotocol.io/) server (available since version 0.7.0), so AI assistants such as Claude Code, Codex, Claude Desktop, and Cursor can analyze audio and video files for you. -The SDK includes an [MCP](https://modelcontextprotocol.io/) server (available since version 0.7.0), so AI assistants such as Claude Code, Codex, Claude Desktop and Cursor can analyze audio and video files for you. The setups below start the server with `uvx`, so install [uv](https://docs.astral.sh/uv/) first. -The server reads your credentials from `BEHAVIORALSIGNALS_CID` and `BEHAVIORALSIGNALS_API_KEY` (see [API Key Setup](#api-key-setup)). -Your assistant starts the server, so set them in the assistant's config, as shown below. +The server reads credentials from `BEHAVIORALSIGNALS_CID` and `BEHAVIORALSIGNALS_API_KEY`. Your assistant starts the server, so provide them in the assistant's MCP configuration as shown below. -**Claude Code.** With the two variables set in your shell, run: +### Claude Code + +With the two variables set in your shell, run: ```bash claude mcp add --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \ @@ -326,9 +335,11 @@ claude mcp add --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \ behavioralsignals -- uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp ``` -This adds the server for you in the current project only. Your key stays out of your shell history and out of the repository. +This adds the server to the current project only. Your key stays out of your shell history and out of the repository. + +### Codex -**Codex.** With the two variables set in your shell, run: +With the two variables set in your shell, run: ```bash codex mcp add behavioralsignals --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \ @@ -336,16 +347,24 @@ codex mcp add behavioralsignals --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_ -- uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp ``` -This adds the server for all your projects, in `~/.codex/config.toml`. +This adds the server for all your projects in `~/.codex/config.toml`. -**Claude Desktop and Cursor.** Add this to `claude_desktop_config.json` (in Claude Desktop, Settings > Developer > Edit Config) or to `~/.cursor/mcp.json`: +### Claude Desktop and Cursor + +Add this to `claude_desktop_config.json` (Claude Desktop: **Settings > Developer > Edit Config**) or `~/.cursor/mcp.json`: ```json { "mcpServers": { "behavioralsignals": { "command": "uvx", - "args": ["--python", ">=3.10", "--from", "behavioralsignals[mcp]", "behavioralsignals-mcp"], + "args": [ + "--python", + ">=3.10", + "--from", + "behavioralsignals[mcp]", + "behavioralsignals-mcp" + ], "env": { "BEHAVIORALSIGNALS_CID": "your_cid", "BEHAVIORALSIGNALS_API_KEY": "your_api_key" @@ -355,32 +374,63 @@ This adds the server for all your projects, in `~/.codex/config.toml`. } ``` -If the app cannot find `uvx`, use its full path (run `which uvx` to find it). +If the app cannot find `uvx`, use its full path (`which uvx`). + +### MCP Tools -The server has 4 tools: +The server exposes four tools: | Tool | What it does | |---|---| | `analyze_behavior` | Uploads an audio file or S3 presigned URL for behavioral analysis and returns the results | | `detect_deepfake` | Uploads an audio or video file, or an S3 presigned URL, for deepfake detection and returns the results | -| `get_result` | Returns the results of a process, in pages, optionally only for some tasks | -| `list_processes` | Lists your processes, newest first, with the reason when one failed | +| `get_result` | Returns the results of a process, in pages, optionally filtered to specific tasks | +| `list_processes` | Lists processes, newest first, including the failure reason when available | Notes: -* Each upload uses credits, and the files you name are sent to the Behavioral Signals API. -* The upload tools wait up to `wait_seconds` (45 by default, 50 at most). If processing takes longer, they return the process ID, and the assistant checks it later with `get_result`. -* Don't commit config files that contain your API key. -### Running the MCP server manually +- Each upload uses API credits, and files named in MCP tool calls are sent to the Behavioral Signals API. +- Upload tools wait up to `wait_seconds` (45 seconds by default, 50 seconds at most). If processing takes longer, they return the process ID so the assistant can check it later with `get_result`. +- Do not commit configuration files containing your API key. + +### Running the MCP Server Manually -With uv installed, you can run it without installing anything else: +With `uv` installed, run the server without installing the package: ```bash uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp ``` -Or install it with pip and run `behavioralsignals-mcp`: +Or install the MCP extra with pip: ```bash pip install "behavioralsignals[mcp]" +behavioralsignals-mcp ``` + +## Requirements + +- Python 3.10+ +- A Behavioral Signals account and API key + +`ffmpeg` is needed by `make_audio_stream()` for formats other than WAV, such as mp3. + +## Documentation and Examples + +- [Python SDK Documentation](https://behavioralsignals.readme.io/docs/behavioral-signals-python-sdk) +- [Full API Documentation](https://behavioralsignals.readme.io/) +- [Streaming with the Python SDK](https://behavioralsignals.readme.io/docs/streaming-using-python-sdk) +- [Video Deepfake Detection](https://behavioralsignals.readme.io/docs/submit-a-file-for-processing) +- [Batch Examples](examples/batch/) +- [Streaming Examples](examples/streaming/) +- [AI-friendly Documentation (`llms.txt`)](https://behavioralsignals.readme.io/llms.txt) + +## Contributing + +Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, formatting, and pull request guidelines. + +Please report security issues according to [SECURITY.md](SECURITY.md). + +## License + +Licensed under the Apache License 2.0. See [LICENSE](LICENSE). diff --git a/examples/streaming/streaming_api_file.py b/examples/streaming/streaming_api_file.py index e1a8fc6..875ba56 100644 --- a/examples/streaming/streaming_api_file.py +++ b/examples/streaming/streaming_api_file.py @@ -4,7 +4,7 @@ from dotenv import load_dotenv from behavioralsignals import Client, StreamingOptions -from behavioralsignals.utils import make_audio_stream +from behavioralsignals.utils import print_results, make_audio_stream def parse_args(): @@ -54,7 +54,7 @@ def parse_args(): output_data = [] for resp in responses: - print(resp) + print_results(resp.results) resp_data = resp.model_dump() output_data.append(resp_data) diff --git a/pyproject.toml b/pyproject.toml index ec4921c..5d928d4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "behavioralsignals" -version = "0.7.0" +version = "0.7.1" description = "Official Python SDK for the Behavioral Signals API: voice emotion/behavior analysis and audio/video deepfake detection" readme = "README.md" authors = [{ name = "Behavioral Signal Technologies" }] diff --git a/src/behavioralsignals/utils.py b/src/behavioralsignals/utils.py index 80f06f6..18054cb 100644 --- a/src/behavioralsignals/utils.py +++ b/src/behavioralsignals/utils.py @@ -1,7 +1,6 @@ -from collections.abc import Iterator +from collections.abc import Iterable, Iterator -from pydub import AudioSegment -from pydub.utils import make_chunks +from .models import ResultItem def make_audio_stream(file_path: str, chunk_size: float = 0.25) -> tuple[Iterator[bytes], int]: @@ -15,6 +14,9 @@ def make_audio_stream(file_path: str, chunk_size: float = 0.25) -> tuple[Iterato Iterator[bytes]: An iterator yielding raw audio data chunks. int: Sample rate of the audio. """ + # Imported here because pydub warns on import when ffmpeg is missing. + from pydub import AudioSegment + from pydub.utils import make_chunks snd = AudioSegment.from_file(file_path) snd = snd.set_sample_width(2) @@ -22,3 +24,21 @@ def make_audio_stream(file_path: str, chunk_size: float = 0.25) -> tuple[Iterato chunks = iter([chunk.raw_data for chunk in make_chunks(snd, chunk_size * 1000)]) return chunks, snd.frame_rate + + +def print_results(items: Iterable[ResultItem] | None) -> None: + """Print one line per result: start time, end time, task, top label and its probability. + + Continuous tasks (e.g. intensity) have no label, so their score is printed instead. + Rows without a label or score (e.g. the features row, which carries embeddings) are skipped. + + Args: + items (Iterable[ResultItem] | None): Result items, e.g. `result.results`. + """ + for item in items or []: + top = item.prediction[0] if item.prediction else None + label = item.finalLabel or (top.score if top else None) + if not label: + continue + confidence = f" ({float(top.posterior):.1%})" if top and top.posterior else "" + print(f"{item.st} {item.et} {item.task} {label.strip()}{confidence}") diff --git a/tests/test_utils.py b/tests/test_utils.py index feed883..bae75ec 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -1,8 +1,11 @@ +import sys import wave +import subprocess import pytest -from behavioralsignals.utils import make_audio_stream +from behavioralsignals.utils import print_results, make_audio_stream +from behavioralsignals.models import ResultItem @pytest.fixture @@ -23,3 +26,46 @@ def test_make_audio_stream_splits_the_audio_into_chunks(wav_file): assert sample_rate == 16000 assert len(chunks) == 4 assert {len(chunk) for chunk in chunks} == {8000} + + +def _item(task, prediction, final_label=None): + return ResultItem( + task=task, + startTime="0.487", + endTime="3.001", + prediction=prediction, + finalLabel=final_label, + ) + + +def test_print_results_prints_one_line_per_result(capsys): + print_results( + [ + _item("emotion", [{"label": "sad", "posterior": "0.706"}], "sad"), + _item("intensity", [{"score": "0.0873"}]), + _item("diarization", [{"label": "SPEAKER_00"}], "SPEAKER_00"), + _item("features", [{"label": None}]), + _item("gender", None), + _item("language", None, "en"), + _item("asr", None, " Hello there."), + ] + ) + assert capsys.readouterr().out == ( + "0.487 3.001 emotion sad (70.6%)\n" + "0.487 3.001 intensity 0.0873\n" + "0.487 3.001 diarization SPEAKER_00\n" + "0.487 3.001 language en\n" + "0.487 3.001 asr Hello there.\n" + ) + + +def test_print_results_accepts_none(capsys): + print_results(None) + assert capsys.readouterr().out == "" + + +def test_importing_utils_does_not_load_pydub(): + """pydub warns when ffmpeg is missing, so only make_audio_stream() should load it.""" + code = "import sys, behavioralsignals.utils; print('pydub' in sys.modules)" + out = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, check=True) + assert out.stdout.strip() == "False" diff --git a/uv.lock b/uv.lock index f24f1b1..f30946b 100644 --- a/uv.lock +++ b/uv.lock @@ -164,7 +164,7 @@ wheels = [ [[package]] name = "behavioralsignals" -version = "0.7.0" +version = "0.7.1" source = { editable = "." } dependencies = [ { name = "audioop-lts", marker = "python_full_version >= '3.13'" },