diff --git a/README.md b/README.md index 020c81b..5a69d22 100644 --- a/README.md +++ b/README.md @@ -40,15 +40,6 @@ The diagram below illustrates the RPS application architecture. During algorithm 1. [Create the input interface](./Documentation/README.md#input-interface-and-signal-conditioning), add signal conditioning, and start capturing data for ML model training. 2. [Select an ML model](./Documentation/README.md#create-ml-model), then use the captured data for training, analysis, and creation of the optimized ML model. 3. [Integrate the ML model](./Documentation/README.md#integrate-ml-model) into the SDS framework and analyze performance. -4. Configure `OUTPUT_PREDICTION_METADATA` based on your workflow: - - **Configuration file:** - `RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml` - - Set `OUTPUT_PREDICTION_METADATA = 0` to view the generated `.sds` files using the Arm SDS VS Code extension. - - Set `OUTPUT_PREDICTION_METADATA = 1` to enable live inference streaming in Fusion Studio, which parses the prediction metadata (predicted class label, confidence score, and class index) to render overlayed frames. - > [!Note] - > - > By default, `OUTPUT_PREDICTION_METADATA` is set to `0` **Test Embedded Application:** diff --git a/RockPaperScissors/AppKit-E8_USB/README.md b/RockPaperScissors/AppKit-E8_USB/README.md index b15b6c4..dcffec6 100644 --- a/RockPaperScissors/AppKit-E8_USB/README.md +++ b/RockPaperScissors/AppKit-E8_USB/README.md @@ -65,17 +65,21 @@ The AlgorithmTest project demonstrates real-world usage of the SDS Framework on This project supports both recording and playback functionality. By default, the application starts in recording mode. You can switch modes via the SDSIO Server: use `R` key to start recording, `S` key to stop recording, and `P` key to begin playback. -When used in **recording** mode: - -- **Captures on-board camera stream** via SDS recording stream (ML_In..sds file) -- **Executes ML inference** using an object detection ML model -- **Captures algorithm output** via SDS recording stream (ML_Out..sds file) - -When used in **playback** mode: - -- **Replays pre-recorded video stream** via SDS playback stream (ML_In..sds file) -- **Executes ML inference** using an object detection ML model -- **Captures algorithm output** via SDS recording stream (ML_Out..sds file) +When used in **recording** mode: + +- **Captures on-board camera stream** via SDS recording stream (ML_In..sds file) +- **Executes ML inference** using an object detection ML model +- **Captures algorithm output** via SDS recording stream (ML_Out..sds file) +- **Optionally captures prediction result metadata** via SDS recording stream (ML_Result..sds file) +- **Optionally captures raw output tensor** via SDS recording stream (ML_RawOutput..sds file) + +When used in **playback** mode: + +- **Replays pre-recorded video stream** via SDS playback stream (ML_In..sds file) +- **Executes ML inference** using an object detection ML model +- **Captures algorithm output** via SDS recording stream (ML_Out..sds file) +- **Optionally captures prediction result metadata** via SDS recording stream (ML_Result..sds file) +- **Optionally captures raw output tensor** via SDS recording stream (ML_RawOutput..sds file) ### Setup @@ -121,10 +125,11 @@ Model initialized. Ready for inference. ### Recording Test -To execute the **recording** test, just: - -1. Press the **joystick (SW2)** on the board or press `R` key in the SDSIO Server window to start the recording. -2. Press the **joystick (SW2)** again or press `S` key in the SDSIO Server window to stop the recording. +To execute the **recording** test, just: + +1. Optionally press `F` in the SDSIO Server window to enable `ML_Result` and `ML_RawOutput` from the start of the run. +2. Press the **joystick (SW2)** on the board or press `R` key in the SDSIO Server window to start the recording. +3. Press the **joystick (SW2)** again or press `S` key in the SDSIO Server window to stop the recording. **SDSIO Server Output:** @@ -133,17 +138,21 @@ To execute the **recording** test, just: Press Ctrl+C to exit. Starting SDS Control Flags thread. R=record, P=playback, S/s=stop, X/x=terminate, A-H=set flags 0-7, a-h=clear flags 0-7. Starting USB Server... -SDSIO Client USB device connected. -sdsFlags = 0x10000000 -40% idle - : -Record: ML_In (c:\SDS\ML_In.0.sds) -Record: ML_Out (c:\SDS\ML_Out.0.sds) -.............. -sdsFlags = 0x10000000 -. -Closed: ML_In (c:\SDS\ML_In.0.sds) -Closed: ML_Out (c:\SDS\ML_Out.0.sds) +SDSIO Client USB device connected. +sdsFlags = 0x10000020 +sdsFlags = 0x90000020 +40% idle + : +Record: ML_In (c:\SDS\ML_In.0.sds) +Record: ML_Out (c:\SDS\ML_Out.0.sds) +Record: ML_Result (c:\SDS\ML_Result.0.sds) +Record: ML_RawOutput (c:\SDS\ML_RawOutput.0.sds) +.............. +. +Closed: ML_In (c:\SDS\ML_In.0.sds) +Closed: ML_Out (c:\SDS\ML_Out.0.sds) +Closed: ML_Result (c:\SDS\ML_Result.0.sds) +Closed: ML_RawOutput (c:\SDS\ML_RawOutput.0.sds) ``` **Serial Monitor Output:** @@ -155,9 +164,9 @@ Predicted class : UNKNOWN Confidence : 99.51 % 40% idle ... -==== SDS recording stopped -``` -Each run records two files: `ML_In..sds` and `ML_Out..sds` in the directory where SDSIO Server was started. `` is a sequential number. +==== SDS recording stopped +``` +Each run records `ML_In..sds` and `ML_Out..sds` by default. If flag `F` is set before streaming starts, it also records `ML_Result..sds` and `ML_RawOutput..sds` for the full run. Changes to `F/f` during an active recording are ignored for that run; set `F` before starting the next run if optional streams are needed. `` is a sequential number. #### Check SDS Files @@ -180,12 +189,13 @@ Validation passed ### Playback Test -To execute the **playback** test, just: - -1. Press the `P` key in the SDSIO Server window to start the playback. -2. Press the `S` key in the SDSIO Server window to stop the playback. - -The stream `ML_In..sds` is read back and the algorithm processes this data. The stream `ML_Out..p.sds` is written. +To execute the **playback** test, just: + +1. Optionally press `F` in the SDSIO Server window to enable `ML_Result` and `ML_RawOutput` from the start of playback. +2. Press the `P` key in the SDSIO Server window to start the playback. +3. Press the `S` key in the SDSIO Server window to stop the playback. + +The stream `ML_In..sds` is read back and the algorithm processes this data. The stream `ML_Out..p.sds` is written by default. If flag `F` is set before playback starts, `ML_Result..p.sds` and `ML_RawOutput..p.sds` are also written for the full playback run. > Note: > @@ -196,27 +206,32 @@ The stream `ML_In..sds` is read back and the algorithm processes this data. T **SDSIO-Server Output:** ```bash ->sdsio-server.py usb -sdsControl: start playback ('P') -sdsFlags = 0xB0000000 -Playback: ML_In (c:\SDS\ML_In.0.sds) -Record: ML_Out (c:\SDS\ML_Out.0.p.sds) -Closed: ML_In (c:\SDS\ML_In.0.sds) -. -55% idle +>sdsio-server.py usb +sdsControl: start playback ('P') +sdsFlags = 0xB0000020 +Playback: ML_In (c:\SDS\ML_In.0.sds) +Record: ML_Out (c:\SDS\ML_Out.0.p.sds) +Record: ML_Result (c:\SDS\ML_Result.0.p.sds) +Record: ML_RawOutput (c:\SDS\ML_RawOutput.0.p.sds) +... +Closed: ML_In (c:\SDS\ML_In.0.sds) +. +55% idle ... 13% idle .... 11% idle ..... -Closed: ML_Out (c:\SDS\ML_Out.0.p.sds) -sdsFlags = 0x30000000 -57% idle +Closed: ML_Out (c:\SDS\ML_Out.0.p.sds) +Closed: ML_Result (c:\SDS\ML_Result.0.p.sds) +Closed: ML_RawOutput (c:\SDS\ML_RawOutput.0.p.sds) +sdsFlags = 0x30000000 +57% idle ``` -> Note: -> -> ML_Out file recorded during playback should be identical to the one recorded earlier. +> Note: +> +> ML_Out file recorded during playback should be identical to the one recorded earlier. If optional result streams are enabled with flag `F`, ML_Result and ML_RawOutput playback files should also match the corresponding earlier recordings. Use `F` before `P` to include optional streams from the start of playback. ### Key Components @@ -228,7 +243,7 @@ sdsFlags = 0x30000000 **Algorithm Processing** (`algorithm_user.cpp`): -- Initializes ML model and LCD display stream using CMSIS vStream driver -- Executes ML inference (pre-processing, inference, post-processing) -- Copies detection results to output buffer for SDS recording -- Displays frames on LCD with object type with confidence percentage +- Initializes ML model and LCD display stream using CMSIS vStream driver +- Executes ML inference (pre-processing, inference, post-processing) +- Copies class confidence scores, prediction result metadata, and raw output tensors to SDS recording buffers +- Displays frames on LCD with object type with confidence percentage diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml b/RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml index e1cad5d..0139f23 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml @@ -26,13 +26,12 @@ project: - ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE: 0x100000 - C10_USING_CUSTOM_GENERATED_MACROS - ET_NUM_INFERENCES: 1 - - ET_LOG_DUMP_OUTPUT - - ET_DEBUG_BUFFER_SIZE: 0x8000 - - USE_PERFORMANCE_MONITOR - - USE_SEGGER_SYSVIEW - - OUTPUT_PREDICTION_METADATA: 0 - - setups: + - ET_LOG_DUMP_OUTPUT + - ET_DEBUG_BUFFER_SIZE: 0x8000 + - USE_PERFORMANCE_MONITOR + - USE_SEGGER_SYSVIEW + + setups: - setup: Scratch pool for non-simulator not-for-context: +SSE-320-U85 define: diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm.h b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm.h index f904420..ec93079 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm.h +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm.h @@ -50,6 +50,23 @@ extern void ResetAlgorithm (void); */ extern int32_t ExecuteAlgorithm (uint8_t *in_buf, uint32_t in_num, uint8_t *out_buf, uint32_t out_num); +/** + \fn int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num) + \brief Get algorithm result metadata. + \param[out] out_buf pointer to memory buffer for returning algorithm result metadata + \param[in] out_num maximum number of data bytes available in output buffer (in bytes) + \return 0 on success; -1 on error +*/ +extern int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num); + +/** + \fn int32_t GetAlgorithmRawOutputTensor (uint8_t *out_buf, uint32_t out_num) + \brief Get algorithm raw output tensor bytes. + \param[out] out_buf pointer to memory buffer for returning raw output tensor bytes + \param[in] out_num maximum number of data bytes available in output buffer (in bytes) + \return 0 on success; -1 on error +*/ +extern int32_t GetAlgorithmRawOutputTensor (uint8_t *out_buf, uint32_t out_num); #ifdef __cplusplus } #endif diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_config.h b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_config.h index 18a3f83..9b30857 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_config.h +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_config.h @@ -28,11 +28,15 @@ #endif #ifndef ALGO_DATA_OUT_BLOCK_SIZE -#if defined(OUTPUT_PREDICTION_METADATA) && OUTPUT_PREDICTION_METADATA -#define ALGO_DATA_OUT_BLOCK_SIZE (120U) -#else #define ALGO_DATA_OUT_BLOCK_SIZE (MODEL_NUM_CLASSES * sizeof(float)) #endif -#endif +#ifndef ALGO_RESULT_OUT_BLOCK_SIZE +#define ALGO_RESULT_OUT_BLOCK_SIZE (120U) #endif + +#ifndef ALGO_RAW_OUT_BLOCK_SIZE +#define ALGO_RAW_OUT_BLOCK_SIZE (MODEL_NUM_CLASSES * sizeof(float)) +#endif + +#endif diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_user.cpp b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_user.cpp index d7c87a4..11029f5 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_user.cpp +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_user.cpp @@ -303,4 +303,27 @@ int32_t ExecuteAlgorithm(uint8_t *in_buf, uint32_t in_num, #endif return 0; -} \ No newline at end of file +} + +/** + \fn int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num) + \brief Get algorithm result metadata. + \param[out] out_buf pointer to memory buffer for returning algorithm result metadata + \param[in] out_num maximum number of data bytes available in output buffer (in bytes) + \return 0 on success; -1 on error +*/ +int32_t GetAlgorithmResultMetadata(uint8_t *out_buf, uint32_t out_num) { + if (copy_result_metadata(out_buf, out_num) == 0U) { + return -1; + } + + return 0; +} + +int32_t GetAlgorithmRawOutputTensor(uint8_t *out_buf, uint32_t out_num) { + if (copy_raw_output_tensor(out_buf, out_num) == 0U) { + return -1; + } + + return 0; +} diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.cc b/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.cc index 940882f..656f75f 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.cc +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.cc @@ -17,6 +17,7 @@ #include #include #include "RTE_Components.h" +#include "algorithm_config.h" #include "config_video.h" #include "image_processing_func.h" #include CMSIS_device_header @@ -152,6 +153,9 @@ et_tick_ratio_t et_pal_ticks_to_ns_multiplier(void) { */ typedef classification_result_t output_label_t; +static_assert(sizeof(output_label_t) <= ALGO_RESULT_OUT_BLOCK_SIZE, + "Result metadata output block is too small"); + /* ============================================================================ * External Variables * ============================================================================ @@ -176,11 +180,14 @@ char label_name[MAX_LABEL_NAME_LENGTH] = {0}; bool classify_object = false; -output_label_t output_label; - -float class_probs[MODEL_NUM_CLASSES] = {0.0}; - -constexpr int H = IMAGE_HEIGHT; +output_label_t output_label; + +float class_probs[MODEL_NUM_CLASSES] = {0.0}; + +static uint8_t raw_output_tensor_buf[ALGO_RAW_OUT_BLOCK_SIZE] = {0}; +static size_t raw_output_tensor_size = 0U; + +constexpr int H = IMAGE_HEIGHT; constexpr int W = IMAGE_WIDTH; @@ -842,12 +849,14 @@ void log_mem_status(RunnerContext& ctx) { * \brief Print and process model output tensors * \param[in] ctx Runner context */ -void print_outputs(RunnerContext& ctx) -{ - std::vector outputs(ctx.method.value()->outputs_size()); - - Error status = - ctx.method.value()->get_outputs(outputs.data(), outputs.size()); +void print_outputs(RunnerContext& ctx) +{ + std::vector outputs(ctx.method.value()->outputs_size()); + raw_output_tensor_size = 0U; + memset(raw_output_tensor_buf, 0, sizeof(raw_output_tensor_buf)); + + Error status = + ctx.method.value()->get_outputs(outputs.data(), outputs.size()); ET_CHECK(status == Error::Ok); for (int i = 0; i < outputs.size(); ++i) @@ -856,19 +865,44 @@ void print_outputs(RunnerContext& ctx) printf("Output[%d]: Not Tensor\n", i); continue; } - - Tensor tensor = outputs[i].toTensor(); - - if (tensor.scalar_type() != ScalarType::Float && tensor.scalar_type() != ScalarType::Char) { - continue; - } - - postprocess_data_t result = {0}; - const float* logits = tensor.const_data_ptr(); - int numel = tensor.numel(); - - // Safety check - if (numel != MODEL_NUM_CLASSES) { + + Tensor tensor = outputs[i].toTensor(); + + if (tensor.scalar_type() != ScalarType::Float && tensor.scalar_type() != ScalarType::Char) { + continue; + } + + postprocess_data_t result = {0}; + int numel = tensor.numel(); + const void *raw_tensor_data = nullptr; + size_t raw_tensor_size = 0U; + + if (tensor.scalar_type() == ScalarType::Float) { + raw_tensor_data = tensor.const_data_ptr(); + raw_tensor_size = (size_t)numel * sizeof(float); + } else { + raw_tensor_data = tensor.const_data_ptr(); + raw_tensor_size = (size_t)numel * sizeof(int8_t); + } + + if (raw_tensor_data != nullptr) { + size_t remaining = sizeof(raw_output_tensor_buf) - raw_output_tensor_size; + size_t copy_size = (raw_tensor_size <= remaining) ? raw_tensor_size : remaining; + + if (copy_size > 0U) { + memcpy(&raw_output_tensor_buf[raw_output_tensor_size], raw_tensor_data, copy_size); + raw_output_tensor_size += copy_size; + } + } + + if (tensor.scalar_type() != ScalarType::Float) { + continue; + } + + const float* logits = tensor.const_data_ptr(); + + // Safety check + if (numel != MODEL_NUM_CLASSES) { printf("Error: Output class count mismatch!\n"); printf("Number of classes: %d, expected: %d\n", numel, MODEL_NUM_CLASSES); } @@ -952,16 +986,10 @@ void postprocess(RunnerContext& ctx, uint8_t* img_buf, /* Decode output tensor → output_label, conf_int, classify_object */ print_outputs(ctx); - /* Copy shortened label plus confidence into caller's output buffer */ -#if OUTPUT_PREDICTION_METADATA - if (out_num >= sizeof(output_label_t)) { - memcpy(out_buf, &output_label, sizeof(output_label)); - } -#else + /* Copy class confidence scores into caller's output buffer. */ if (out_num >= sizeof(class_probs)) { memcpy(out_buf, class_probs, sizeof(class_probs)); } -#endif /* Only draw if label is valid */ if (output_label.label_name[0] != '\0') @@ -975,6 +1003,40 @@ void postprocess(RunnerContext& ctx, uint8_t* img_buf, } } +/** + \fn size_t copy_result_metadata (uint8_t *out_buf, size_t out_num) + \brief Copy algorithm result metadata to the output buffer. + \param[out] out_buf pointer to memory buffer for returning algorithm result metadata + \param[in] out_num number of data bytes available in output buffer (in bytes) + \return number of metadata bytes copied on success; 0 on error +*/ +size_t copy_result_metadata(uint8_t *out_buf, size_t out_num) { + if (out_buf == nullptr || out_num < sizeof(output_label)) { + return 0U; + } + + memset(out_buf, 0, out_num); + memcpy(out_buf, &output_label, sizeof(output_label)); + return sizeof(output_label); +} + +/** + \fn size_t copy_raw_output_tensor (uint8_t *out_buf, size_t out_num) + \brief Copy raw output tensor bytes to the output buffer. + \param[out] out_buf pointer to memory buffer for returning raw output tensor bytes + \param[in] out_num number of data bytes available in output buffer (in bytes) + \return number of raw tensor bytes copied on success; 0 on error +*/ +size_t copy_raw_output_tensor(uint8_t *out_buf, size_t out_num) { + if (out_buf == nullptr || out_num < raw_output_tensor_size) { + return 0U; + } + + memset(out_buf, 0, out_num); + memcpy(out_buf, raw_output_tensor_buf, raw_output_tensor_size); + return raw_output_tensor_size; +} + void write_etdump(RunnerContext& ctx) {} /** @@ -1046,4 +1108,4 @@ bool run_inference(RunnerContext& ctx) { ctx.method_name, status); return (status == Error::Ok); -} \ No newline at end of file +} diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.h b/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.h index 3ee52df..70000ed 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.h +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/arm_executor_runner.h @@ -117,4 +117,25 @@ bool run_inference(RunnerContext &ctx); void postprocess(RunnerContext &ctx, uint8_t *img_buf, uint32_t img_width, uint32_t img_height, uint8_t *out_buf, uint32_t out_num); -#endif /* ARM_EXECUTOR_RUNNER_H */ \ No newline at end of file +/** + * \brief Copy the latest classification metadata for the result SDS stream. + * + * The destination buffer is cleared first so callers may use a fixed-size stream + * block that is larger than classification_result_t. + * + * \param[out] out_buf Destination buffer. + * \param[in] out_num Destination buffer size in bytes. + * \return Number of bytes copied from classification_result_t, or 0 on error. + */ +size_t copy_result_metadata(uint8_t *out_buf, size_t out_num); + +/** + \fn size_t copy_raw_output_tensor (uint8_t *out_buf, size_t out_num) + \brief Copy raw output tensor bytes to the output buffer. + \param[out] out_buf pointer to memory buffer for returning raw output tensor bytes + \param[in] out_num number of data bytes available in output buffer (in bytes) + \return number of raw tensor bytes copied on success; 0 on error +*/ +size_t copy_raw_output_tensor(uint8_t *out_buf, size_t out_num); + +#endif /* ARM_EXECUTOR_RUNNER_H */ diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/sds_main.c b/RockPaperScissors/AppKit-E8_USB/algorithm/sds_main.c index 3c041f8..2ebe537 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/sds_main.c +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/sds_main.c @@ -24,6 +24,7 @@ #include "data_in.h" #include "app_setup.h" +#define SDS_FLAG_RECORD_EXTRA_OUTPUTS (1UL << 5) // SDS system state consistent with main AlgorithmThread loop uint32_t sds_state = SDS_STATE_INACTIVE; @@ -35,18 +36,24 @@ uint32_t timeslot = 0U; uint8_t record_camera = 0U; // Algorithm input/output data buffer -static uint8_t algo_data_in_buf [ALGO_DATA_IN_BLOCK_SIZE] __ALIGNED(4); -static uint8_t algo_data_out_buf[ALGO_DATA_OUT_BLOCK_SIZE] __ALIGNED(4); +static uint8_t algo_data_in_buf [ALGO_DATA_IN_BLOCK_SIZE] __ALIGNED(4); +static uint8_t algo_data_out_buf [ALGO_DATA_OUT_BLOCK_SIZE] __ALIGNED(4); +static uint8_t algo_result_out_buf [ALGO_RESULT_OUT_BLOCK_SIZE] __ALIGNED(4); +static uint8_t algo_raw_out_buf [ALGO_RAW_OUT_BLOCK_SIZE] __ALIGNED(4); // SDS buffers -static uint8_t sds_camera_buf [CAMERA_FRAME_SIZE + 2048] __ALIGNED(4); -static uint8_t sds_data_in_buf [ALGO_DATA_IN_BLOCK_SIZE + 2048] __ALIGNED(4); -static uint8_t sds_data_out_buf[(ALGO_DATA_OUT_BLOCK_SIZE * 2) + 2048] __ALIGNED(4); +static uint8_t sds_camera_buf [CAMERA_FRAME_SIZE + 2048] __ALIGNED(4); +static uint8_t sds_data_in_buf [ALGO_DATA_IN_BLOCK_SIZE + 2048] __ALIGNED(4); +static uint8_t sds_data_out_buf [(ALGO_DATA_OUT_BLOCK_SIZE * 2) + 2048] __ALIGNED(4); +static uint8_t sds_result_out_buf [(ALGO_RESULT_OUT_BLOCK_SIZE * 2) + 2048] __ALIGNED(4); +static uint8_t sds_raw_out_buf [(ALGO_RAW_OUT_BLOCK_SIZE * 2) + 2048] __ALIGNED(4); // SDS stream identifiers - sdsId_t sds_camera_id = NULL; -static sdsId_t sds_data_in_id = NULL; -static sdsId_t sds_data_out_id = NULL; + sdsId_t sds_camera_id = NULL; +static sdsId_t sds_data_in_id = NULL; +static sdsId_t sds_data_out_id = NULL; +static sdsId_t sds_result_out_id = NULL; +static sdsId_t sds_raw_out_id = NULL; // Recording/playback mode text static const char *SDS_MODE[] = { "recording", "playback" }; @@ -62,6 +69,7 @@ int32_t OpenStreams (void) { int32_t status = 0; uint8_t play = 0U; uint8_t camera_fail = 0U; + uint8_t extra_outs_fail = 0U; if ((sdsFlags & SDS_FLAG_PLAYBACK) != 0U) { // If open for playback requested play = 1U; @@ -96,7 +104,23 @@ int32_t OpenStreams (void) { SDS_ASSERT(sds_data_in_id != NULL); SDS_ASSERT(sds_data_out_id != NULL); - if ((camera_fail == 0U) && (sds_data_in_id != NULL) && (sds_data_out_id != NULL)) { + // Optional output streams are session-level streams. Flag F must be set before start. + if ((sdsFlags & SDS_FLAG_RECORD_EXTRA_OUTPUTS) != 0U) { + sdsFlagsModify(0U, SDS_FLAG_RECORD_EXTRA_OUTPUTS); + + sds_result_out_id = sdsOpen("ML_Result", sdsModeWrite, sds_result_out_buf, sizeof(sds_result_out_buf)); + sds_raw_out_id = sdsOpen("ML_RawOutput", sdsModeWrite, sds_raw_out_buf, sizeof(sds_raw_out_buf)); + + SDS_ASSERT(sds_result_out_id != NULL); + SDS_ASSERT(sds_raw_out_id != NULL); + + if ((sds_result_out_id == NULL) || (sds_raw_out_id == NULL)) { + extra_outs_fail = 1U; + } + } + + if ((camera_fail == 0U) && (extra_outs_fail == 0U) && + (sds_data_in_id != NULL) && (sds_data_out_id != NULL)) { SDS_PRINTF("==== SDS %s started\n", SDS_MODE[play]); } else { sdsState = SDS_STATE_END; // If files could not be opened then request streaming end @@ -148,6 +172,24 @@ int32_t CloseStreams (void) { status = -1; } } + if (sds_result_out_id != NULL) { + close_status = sdsClose(sds_result_out_id); + SDS_ERROR_CHECK(close_status); + if (close_status == SDS_OK) { + sds_result_out_id = NULL; + } else { + status = -1; + } + } + if (sds_raw_out_id != NULL) { + close_status = sdsClose(sds_raw_out_id); + SDS_ERROR_CHECK(close_status); + if (close_status == SDS_OK) { + sds_raw_out_id = NULL; + } else { + status = -1; + } + } if (status == 0) { SDS_PRINTF("==== SDS %s stopped\n", SDS_MODE[play]); @@ -196,7 +238,7 @@ __NO_RETURN void AlgorithmThread (void *argument) { do { ret = sdsRead(sds_data_in_id, ×lot, algo_data_in_buf, sizeof(algo_data_in_buf)); if (ret == SDS_NO_DATA) { - osDelay(1U); + osDelay(1U); DiscardInputData(); } } while (ret == SDS_NO_DATA); @@ -247,6 +289,33 @@ __NO_RETURN void AlgorithmThread (void *argument) { } } while (ret == SDS_NO_SPACE); SDS_ASSERT(ret == sizeof(algo_data_out_buf)); + + if ((sds_result_out_id != NULL) && (sds_raw_out_id != NULL)) { + if (GetAlgorithmResultMetadata(algo_result_out_buf, sizeof(algo_result_out_buf)) != 0) { + continue; + } + if (GetAlgorithmRawOutputTensor(algo_raw_out_buf, sizeof(algo_raw_out_buf)) != 0) { + continue; + } + + // Record prediction result metadata + do { + ret = sdsWrite(sds_result_out_id, timeslot, algo_result_out_buf, sizeof(algo_result_out_buf)); + if (ret == SDS_NO_SPACE) { + osDelay(1U); + } + } while (ret == SDS_NO_SPACE); + SDS_ASSERT(ret == sizeof(algo_result_out_buf)); + + // Record raw output tensor + do { + ret = sdsWrite(sds_raw_out_id, timeslot, algo_raw_out_buf, sizeof(algo_raw_out_buf)); + if (ret == SDS_NO_SPACE) { + osDelay(1U); + } + } while (ret == SDS_NO_SPACE); + SDS_ASSERT(ret == sizeof(algo_raw_out_buf)); + } } } }