Add Support for Saving Predicted ML Metadata and Raw Output Tensors to SDS Files

README.md

@@ -40,15 +40,6 @@ The diagram below illustrates the RPS application architecture. During algorithm
 1. [Create the input interface](./Documentation/README.md#input-interface-and-signal-conditioning), add signal conditioning, and start capturing data for ML model training.
 2. [Select an ML model](./Documentation/README.md#create-ml-model), then use the captured data for training, analysis, and creation of the optimized ML model.
 3. [Integrate the ML model](./Documentation/README.md#integrate-ml-model) into the SDS framework and analyze performance.
-4. Configure `OUTPUT_PREDICTION_METADATA` based on your workflow:
-
-   **Configuration file:**
-   `RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml`
-   - Set `OUTPUT_PREDICTION_METADATA = 0` to view the generated `.sds` files using the Arm SDS VS Code extension.
-   - Set `OUTPUT_PREDICTION_METADATA = 1` to enable live inference streaming in Fusion Studio, which parses the prediction metadata (predicted class label, confidence score, and class index) to render overlayed frames.
-    > [!Note]
-    >
-    > By default, `OUTPUT_PREDICTION_METADATA` is set to `0`
 
 **Test Embedded Application:**
 

RockPaperScissors/AppKit-E8_USB/README.md

@@ -65,17 +65,21 @@ The AlgorithmTest project demonstrates real-world usage of the SDS Framework on
 This project supports both recording and playback functionality. By default, the application starts in recording mode.
 You can switch modes via the SDSIO Server: use `R` key to start recording, `S` key to stop recording, and `P` key to begin playback.
 
-When used in **recording** mode:
-
-- **Captures on-board camera stream** via SDS recording stream (ML_In.<n>.sds file)
-- **Executes ML inference** using an object detection ML model
-- **Captures algorithm output** via SDS recording stream (ML_Out.<n>.sds file)
-
-When used in **playback** mode:
-
-- **Replays pre-recorded video stream** via SDS playback stream (ML_In.<n>.sds file)
-- **Executes ML inference** using an object detection ML model
-- **Captures algorithm output** via SDS recording stream (ML_Out.<m>.sds file)
+When used in **recording** mode:
+
+- **Captures on-board camera stream** via SDS recording stream (ML_In.<n>.sds file)
+- **Executes ML inference** using an object detection ML model
+- **Captures algorithm output** via SDS recording stream (ML_Out.<n>.sds file)
+- **Optionally captures prediction result metadata** via SDS recording stream (ML_Result.<n>.sds file)
+- **Optionally captures raw output tensor** via SDS recording stream (ML_RawOutput.<n>.sds file)
+
+When used in **playback** mode:
+
+- **Replays pre-recorded video stream** via SDS playback stream (ML_In.<n>.sds file)
+- **Executes ML inference** using an object detection ML model
+- **Captures algorithm output** via SDS recording stream (ML_Out.<m>.sds file)
+- **Optionally captures prediction result metadata** via SDS recording stream (ML_Result.<m>.sds file)
+- **Optionally captures raw output tensor** via SDS recording stream (ML_RawOutput.<m>.sds file)
 
 ### Setup
 
@@ -121,10 +125,11 @@ Model initialized. Ready for inference.
 
 ### Recording Test
 
-To execute the **recording** test, just:
-
-1. Press the **joystick (SW2)** on the board or press `R` key in the SDSIO Server window to start the recording.
-2. Press the **joystick (SW2)** again or press `S` key in the SDSIO Server window to stop the recording.
+To execute the **recording** test, just:
+
+1. Optionally press `F` in the SDSIO Server window to enable `ML_Result` and `ML_RawOutput` from the start of the run.
+2. Press the **joystick (SW2)** on the board or press `R` key in the SDSIO Server window to start the recording.
+3. Press the **joystick (SW2)** again or press `S` key in the SDSIO Server window to stop the recording.
 
 **SDSIO Server Output:**
 
@@ -133,17 +138,21 @@ To execute the **recording** test, just:
 Press Ctrl+C to exit.
 Starting SDS Control Flags thread. R=record, P=playback, S/s=stop, X/x=terminate, A-H=set flags 0-7, a-h=clear flags 0-7.
 Starting USB Server...
-SDSIO Client USB device connected.
-sdsFlags = 0x10000000
-40% idle
- :
-Record:   ML_In (c:\SDS\ML_In.0.sds)
-Record:   ML_Out (c:\SDS\ML_Out.0.sds)
-..............
-sdsFlags = 0x10000000
-.
-Closed:   ML_In (c:\SDS\ML_In.0.sds)
-Closed:   ML_Out (c:\SDS\ML_Out.0.sds)
+SDSIO Client USB device connected.
+sdsFlags = 0x10000020
+sdsFlags = 0x90000020
+40% idle
+ :
+Record:   ML_In (c:\SDS\ML_In.0.sds)
+Record:   ML_Out (c:\SDS\ML_Out.0.sds)
+Record:   ML_Result (c:\SDS\ML_Result.0.sds)
+Record:   ML_RawOutput (c:\SDS\ML_RawOutput.0.sds)
+..............
+.
+Closed:   ML_In (c:\SDS\ML_In.0.sds)
+Closed:   ML_Out (c:\SDS\ML_Out.0.sds)
+Closed:   ML_Result (c:\SDS\ML_Result.0.sds)
+Closed:   ML_RawOutput (c:\SDS\ML_RawOutput.0.sds)
 ```
 
 **Serial Monitor Output:**
@@ -155,9 +164,9 @@ Predicted class : UNKNOWN
 Confidence      : 99.51 %
 40% idle
 ...
-==== SDS recording stopped
-```
-Each run records two files: `ML_In.<n>.sds` and `ML_Out.<n>.sds` in the directory where SDSIO Server was started. `<n>` is a sequential number.
+==== SDS recording stopped
+```
+Each run records `ML_In.<n>.sds` and `ML_Out.<n>.sds` by default. If flag `F` is set before streaming starts, it also records `ML_Result.<n>.sds` and `ML_RawOutput.<n>.sds` for the full run. Changes to `F/f` during an active recording are ignored for that run; set `F` before starting the next run if optional streams are needed. `<n>` is a sequential number.
 
 #### Check SDS Files
 
@@ -180,12 +189,13 @@ Validation passed
 
 ### Playback Test
 
-To execute the **playback** test, just:
-
-1. Press the `P` key in the SDSIO Server window to start the playback.
-2. Press the `S` key in the SDSIO Server window to stop the playback.
-
-The stream `ML_In.<n>.sds` is read back and the algorithm processes this data. The stream `ML_Out.<n>.p.sds` is written.
+To execute the **playback** test, just:
+
+1. Optionally press `F` in the SDSIO Server window to enable `ML_Result` and `ML_RawOutput` from the start of playback.
+2. Press the `P` key in the SDSIO Server window to start the playback.
+3. Press the `S` key in the SDSIO Server window to stop the playback.
+
+The stream `ML_In.<n>.sds` is read back and the algorithm processes this data. The stream `ML_Out.<n>.p.sds` is written by default. If flag `F` is set before playback starts, `ML_Result.<n>.p.sds` and `ML_RawOutput.<n>.p.sds` are also written for the full playback run.
 
 > Note:
 >
@@ -196,27 +206,32 @@ The stream `ML_In.<n>.sds` is read back and the algorithm processes this data. T
 **SDSIO-Server Output:**
 
 ```bash
->sdsio-server.py usb
-sdsControl: start playback ('P')
-sdsFlags = 0xB0000000
-Playback: ML_In (c:\SDS\ML_In.0.sds)
-Record:   ML_Out (c:\SDS\ML_Out.0.p.sds)
-Closed:   ML_In (c:\SDS\ML_In.0.sds)
-.
-55% idle
+>sdsio-server.py usb
+sdsControl: start playback ('P')
+sdsFlags = 0xB0000020
+Playback: ML_In (c:\SDS\ML_In.0.sds)
+Record:   ML_Out (c:\SDS\ML_Out.0.p.sds)
+Record:   ML_Result (c:\SDS\ML_Result.0.p.sds)
+Record:   ML_RawOutput (c:\SDS\ML_RawOutput.0.p.sds)
+...
+Closed:   ML_In (c:\SDS\ML_In.0.sds)
+.
+55% idle
 ...
 13% idle
 ....
 11% idle
 .....
-Closed:   ML_Out (c:\SDS\ML_Out.0.p.sds)
-sdsFlags = 0x30000000
-57% idle
+Closed:   ML_Out (c:\SDS\ML_Out.0.p.sds)
+Closed:   ML_Result (c:\SDS\ML_Result.0.p.sds)
+Closed:   ML_RawOutput (c:\SDS\ML_RawOutput.0.p.sds)
+sdsFlags = 0x30000000
+57% idle
 ```
 
-> Note:
->
-> ML_Out file recorded during playback should be identical to the one recorded earlier.
+> Note:
+>
+> ML_Out file recorded during playback should be identical to the one recorded earlier. If optional result streams are enabled with flag `F`, ML_Result and ML_RawOutput playback files should also match the corresponding earlier recordings. Use `F` before `P` to include optional streams from the start of playback.
 
 ### Key Components
 
@@ -228,7 +243,7 @@ sdsFlags = 0x30000000
 
 **Algorithm Processing** (`algorithm_user.cpp`):
 
-- Initializes ML model and LCD display stream using CMSIS vStream driver
-- Executes ML inference (pre-processing, inference, post-processing)
-- Copies detection results to output buffer for SDS recording
-- Displays frames on LCD with object type with confidence percentage
+- Initializes ML model and LCD display stream using CMSIS vStream driver
+- Executes ML inference (pre-processing, inference, post-processing)
+- Copies class confidence scores, prediction result metadata, and raw output tensors to SDS recording buffers
+- Displays frames on LCD with object type with confidence percentage

RockPaperScissors/AppKit-E8_USB/algorithm/AlgorithmTest.cproject.yml

@@ -26,13 +26,12 @@ project:
     - ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE: 0x100000
     - C10_USING_CUSTOM_GENERATED_MACROS
     - ET_NUM_INFERENCES: 1
-    - ET_LOG_DUMP_OUTPUT
-    - ET_DEBUG_BUFFER_SIZE: 0x8000 
-    - USE_PERFORMANCE_MONITOR
-    - USE_SEGGER_SYSVIEW
-    - OUTPUT_PREDICTION_METADATA: 0
-
-  setups:
+    - ET_LOG_DUMP_OUTPUT
+    - ET_DEBUG_BUFFER_SIZE: 0x8000
+    - USE_PERFORMANCE_MONITOR
+    - USE_SEGGER_SYSVIEW
+
+  setups:
     - setup: Scratch pool for non-simulator
       not-for-context: +SSE-320-U85
       define:

RockPaperScissors/AppKit-E8_USB/algorithm/algorithm.h

@@ -50,6 +50,23 @@ extern void ResetAlgorithm (void);
 */
 extern int32_t ExecuteAlgorithm (uint8_t *in_buf, uint32_t in_num, uint8_t *out_buf, uint32_t out_num);
 
+/**
+  \fn           int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num)
+  \brief        Get algorithm result metadata.
+  \param[out]   out_buf         pointer to memory buffer for returning algorithm result metadata
+  \param[in]    out_num         maximum number of data bytes available in output buffer (in bytes)
+  \return       0 on success; -1 on error
+*/
+extern int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num);
+
+/**
+  \fn           int32_t GetAlgorithmRawOutputTensor (uint8_t *out_buf, uint32_t out_num)
+  \brief        Get algorithm raw output tensor bytes.
+  \param[out]   out_buf         pointer to memory buffer for returning raw output tensor bytes
+  \param[in]    out_num         maximum number of data bytes available in output buffer (in bytes)
+  \return       0 on success; -1 on error
+*/
+extern int32_t GetAlgorithmRawOutputTensor (uint8_t *out_buf, uint32_t out_num);
 #ifdef  __cplusplus
 }
 #endif

RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_config.h

@@ -28,11 +28,15 @@
 #endif
 
 #ifndef ALGO_DATA_OUT_BLOCK_SIZE
-#if defined(OUTPUT_PREDICTION_METADATA) && OUTPUT_PREDICTION_METADATA
-#define ALGO_DATA_OUT_BLOCK_SIZE    (120U)
-#else
 #define ALGO_DATA_OUT_BLOCK_SIZE    (MODEL_NUM_CLASSES * sizeof(float))
 #endif
-#endif
 
+#ifndef ALGO_RESULT_OUT_BLOCK_SIZE
+#define ALGO_RESULT_OUT_BLOCK_SIZE  (120U)
 #endif
+
+#ifndef ALGO_RAW_OUT_BLOCK_SIZE
+#define ALGO_RAW_OUT_BLOCK_SIZE     (MODEL_NUM_CLASSES * sizeof(float))
+#endif
+
+#endif

RockPaperScissors/AppKit-E8_USB/algorithm/algorithm_user.cpp

@@ -303,4 +303,27 @@ int32_t ExecuteAlgorithm(uint8_t *in_buf, uint32_t in_num,
 #endif
 
     return 0;
-}
+}
+
+/**
+  \fn           int32_t GetAlgorithmResultMetadata (uint8_t *out_buf, uint32_t out_num)
+  \brief        Get algorithm result metadata.
+  \param[out]   out_buf         pointer to memory buffer for returning algorithm result metadata
+  \param[in]    out_num         maximum number of data bytes available in output buffer (in bytes)
+  \return       0 on success; -1 on error
+*/
+int32_t GetAlgorithmResultMetadata(uint8_t *out_buf, uint32_t out_num) {
+    if (copy_result_metadata(out_buf, out_num) == 0U) {
+        return -1;
+    }
+
+    return 0;
+}
+
+int32_t GetAlgorithmRawOutputTensor(uint8_t *out_buf, uint32_t out_num) {
+    if (copy_raw_output_tensor(out_buf, out_num) == 0U) {
+        return -1;
+    }
+
+    return 0;
+}