DriverValidation.txt

/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\mainpage Overview

This manual explains the scope and the usage of the <b>CMSIS-Driver Validation</b> framework. This is a test suite that helps developers to verify that an implementation of a peripheral driver is compliant with the corresponding **[CMSIS-Driver Specification](https://arm-software.github.io/CMSIS_5/Driver/html/index.html)**. Verified drivers can then be reliably used with middleware components and user applications that rely on CMSIS-Driver APIs.

The CMSIS-Driver Validation is maintained in a public **[GitHub repository](https://github.com/ARM-software/CMSIS-Driver_Validation)**. Its releases in **[CMSIS Pack format](https://www.open-cmsis-pack.org/)** are also available on **[CMSIS Packs page](https://developer.arm.com/tools-and-software/embedded/cmsis/cmsis-packs)** under *Arm* - *CMSIS Driver Validation* category and can be used in environments supporting the CMSIS-Pack concept.

The CMSIS-Driver Validation framework provides:
  - Configurable validation tests for various CMSIS-Driver interfaces
  - Example projects that show the usage of the CMSIS-Driver Validation
  - Various Servers used for testing

The CMSIS-Driver Validation tests and verifies:
  - <b>API interface</b> using the driver capabilities as well as valid and invalid parameters
  - <b>Data communication</b> with various transfer sizes and communication parameters:
    - <b>Loopback testing</b> (for some interfaces) for testing of the underlying hardware with usage of a local loopback
    - <b>Server testing</b> (for some interfaces) for extensive testing of the underlying hardware with usage of a dedicated Server
  - <b>Transfer speed</b> of the data communication with time measurement of data transfer duration
  - <b>Event</b> signaling

The CMSIS-Driver Validation requires
<a href="https://arm-software.github.io/CMSIS_5/RTOS2/html/index.html" target="_blank">CMSIS-RTOS2</a> functionality and can be used to verify the setup and configuration of the CMSIS-Driver interfaces in a user system.

The diagram below shows an overview of the CMSIS-Driver Validation configuration.

\image html cmsis_dv.png width=100%

The CMSIS-Driver Validation provides validation for the following interfaces:
  - \ref dv_can   "CAN"        - Controller Area Network (CAN) interface driver.
  - \ref dv_eth   "Ethernet"   - Ethernet MAC and PHY peripheral interface driver.
  - \ref dv_gpio  "GPIO"       - General Purpose Input-Output interface driver.
  - \ref dv_i2c   "I2C"        - Inter-Integrated Circuit (I2C) multi-master serial single-ended bus interface driver.
  - \ref dv_mci   "MCI"        - Memory Card Interface driver for SD/MMC memory.
  - \ref dv_spi   "SPI"        - Serial Peripheral Interface (SPI) driver.
  - \ref dv_usart "USART"      - Universal Synchronous and Asynchronous Receiver/Transmitter (USART) interface driver.
  - \ref dv_usbd  "USB Device" - Universal Serial Bus (USB) Device interface driver.
  - \ref dv_usbh  "USB Host"   - Universal Serial Bus (USB) Host interface driver.
  - \ref dv_wifi  "WiFi"       - WiFi (Wireless Fidelity Interface) module/shield driver.

\note Extensive testing using dedicated Server is available for SPI and USART drivers.

This manual assumes that you are familiar with MDK. Refer to
<a href="https://developer.arm.com/documentation/KGS1" target="_blank">MDK Version 5 - Getting Started</a> for additional information.

\section doc_structure Document structure

This manual contains the following chapters:
  - \ref setup                 - Describes the general setup of the CMSIS-Driver Validation test and how to generate test report.
  - \ref report                - Describes the reports produced by the CMSIS-Driver Validation.
  - \ref debugging             - Describes procedure for debugging of the interface drivers using the CMSIS-Driver Validation.
  - \ref resource_requirements - Lists memory and CMSIS-RTOS2 requirements.
  - \ref examples              - Contains information about several example projects including the required hardware setup.
  - <a class="el" href="./topics.html">Reference</a> - Explains the configuration and tests for the various CMSIS-Driver interfaces.

\section License License

The CMSIS Driver example implementations are provided free of charge under Apache 2.0 license.
See the <a href="../../LICENSE.txt">Apache 2.0 License</a>.

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page rev_hist Revision History

The table below provides information about the changes delivered with specific versions of CMSIS-Driver Validation.

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page setup Setup

Using the [Arm CMSIS Solution](https://marketplace.visualstudio.com/items?itemName=Arm.cmsis-csolution) VS Code extension,
create a new solution from **template**:

## Step 1: In the **CMSIS** extension, click **Create a New Solution**.
\image html server_new_sol_create.png

## Step 2: Select the **Target Board**.

## Step 3: Under **Templates, Reference Applications, and Examples**, select **Driver Validation (CMSIS-Driver Validation application)**.
\image html driver_validation_new_sol_template.png

## Step 4: Choose **Solution Base Folder**, click **Create**.
\image html driver_validation_new_sol_create.png

## Step 5: Select the **Board Layer** that provides **required CMSIS Driver**, click **OK**.
\image html driver_validation_new_sol_layer.png

## Step 6: In the **Manage Solution**, under **Build Type** select desired Driver for validation, click **Save**.
\image html driver_validation_build_type.png

## Step 7: Configure the Driver Tests in appropriate **DV_..._Config.h** file.

> **Note:** `...` in **DV_`...`_Config.h** file depends on the driver being validated; for example, for Ethernet, the file would be **DV_ETH_Config.h** file.

For validation configuration of specific driver check respective driver documentation:
- [CAN](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__can__config.html)
- [Ethernet](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__eth__config.html)
- [GPIO](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__gpio__config.html)
- [I2C](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__i2c__config.html)
- [MCI](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__mci__config.html)
- [SPI](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__spi__config.html)
- [USART](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__usart__config.html)
- [USB Device](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__usbd__config.html)
- [USB Host](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__usbh__config.html)
- [WiFi](https://arm-software.github.io/CMSIS-Driver_Validation/latest/group__wifi__config.html)

## Step 8: Setup the required hardware

For the interfaces that support loopback testing: \ref dv_eth "Ethernet", \ref dv_usart "USART" and \ref dv_spi "SPI",
connect the following pins on your target hardware together (refer to the hardware schematics):

- Ethernet: RX+ and TX+, RX- and TX-
- USART:    RX and TX
- SPI:      MOSI and MISO

For the interfaces that support testing with dedicated server: \ref dv_wifi "WiFi", \ref dv_spi "SPI" and \ref dv_usart "USART"
connect the related hardware as required by the related server:

- WiFi: WiFi module has to be in close proximity to the Access Point which is in the same network as the required \ref wifi_sock_setup
- SPI:  MOSI, MISO, SCLK, SS, GND to the same lines on the \ref spi_server
- USART:  Tx, Rx, GND and optionally CLK, CTS, RTS, DCD and RI lines as described in \ref usart_server

## Step 9: Build and Run the application

Example below shows output results (in STDOUT channel) of an Ethernet driver testing:
\verbatim
CMSIS-Driver_Validation v3.1.0 CMSIS-Driver ETH Test Report   Oct  9 2025   07:30:34 

TEST 01: ETH_MAC_GetVersion               
  DV_ETH.c (267): [INFO] API version 2.2, Driver version 3.1
                                          PASSED
TEST 02: ETH_MAC_GetCapabilities          PASSED
TEST 03: ETH_MAC_Initialization           PASSED
TEST 04: ETH_MAC_PowerControl             
  DV_ETH.c (366): [WARNING] Low power is not supported
                                          PASSED
TEST 05: ETH_MAC_MacAddress               PASSED
TEST 06: ETH_MAC_SetBusSpeed              
  DV_ETH.c (445): [WARNING] Link speed 1G is not supported
                                          PASSED
TEST 07: ETH_MAC_Config_Mode              PASSED
TEST 08: ETH_MAC_Config_CommonParams      PASSED
TEST 09: ETH_MAC_Control_Filtering        PASSED
TEST 10: ETH_MAC_SetAddressFilter         PASSED
TEST 11: ETH_MAC_VLAN_Filter              
  DV_ETH.c (910): [WARNING] Received non VLAN tagged frame
                                          PASSED
TEST 12: ETH_MAC_SignalEvent              PASSED
TEST 13: ETH_MAC_PTP_ControlTimer         
  DV_ETH.c (1412): [WARNING] Precision Time Protocol is not supported
                                          NOT EXECUTED
TEST 14: ETH_MAC_CheckInvalidInit         PASSED
TEST 15: ETH_PHY_GetVersion               
  DV_ETH.c (1018): [INFO] API version 2.2, Driver version 1.3
                                          PASSED
TEST 16: ETH_PHY_Initialization           PASSED
TEST 17: ETH_PHY_PowerControl             
  DV_ETH.c (1114): [WARNING] Low power is not supported
  DV_ETH.c (1131): [WARNING] MAC is locked when PHY power is off
                                          PASSED
TEST 18: ETH_PHY_Config                   PASSED
TEST 19: ETH_PHY_CheckInvalidInit         PASSED
TEST 20: ETH_Loopback_Transfer            PASSED
TEST 21: ETH_Loopback_PTP                 
  DV_ETH.c (1574): [WARNING] Precision Time Protocol is not supported
                                          NOT EXECUTED
TEST 22: ETH_Loopback_External            PASSED

Test Summary: 22 Tests, 20 Passed, 0 Failed.
Test Result: PASSED
\endverbatim

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page report Report

The CMSIS-Driver Validation outputs the test report in a <b>Plain Text</b> format.

This report can be viewed using the <b>Debug (printf) Viewer</b> window or any terminal application.

An example of a report captured in a terminal application is shown below: 
\verbatim
CMSIS-Driver_Validation v3.1.0 CMSIS-Driver GPIO Test Report   Oct  9 2025   09:15:16 

TEST 01: GPIO_Setup                       PASSED
TEST 02: GPIO_SetDirection                PASSED
TEST 03: GPIO_SetOutputMode               PASSED
TEST 04: GPIO_SetPullResistor             PASSED
TEST 05: GPIO_SetEventTrigger             PASSED
TEST 06: GPIO_SetOutput                   PASSED
TEST 07: GPIO_GetInput                    PASSED

Test Summary: 7 Tests, 7 Passed, 0 Failed.
Test Result: PASSED

\endverbatim

> **Note:** XML format was deprecated and removed, only <b>Plain Text</b> mode is available now.

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page debugging Debugging

After running the CMSIS-Driver Validation output report is used to see if the driver is compliant to the CMSIS-Driver specification.

If the result of the driver testing under <c>Test Summary</c> contains any <c>Failed</c> tests then the driver needs to be corrected.

Example of report in the Plain Text format of a non-compliant SPI driver:
\verbatim
CMSIS-Driver SPI Test Report   May  6 2020   10:47:11

TEST 01: SPI_GetVersion
  DV_SPI.c (1023): [INFO] Driver API version 2.3, Driver version 2.15
                                          PASSED
TEST 02: SPI_GetCapabilities              PASSED
TEST 03: SPI_Initialize_Uninitialize
  DV_SPI.c (1106): [FAILED]
  DV_SPI.c (1109): [FAILED]
  DV_SPI.c (1112): [FAILED]
                                          FAILED
TEST 04: SPI_PowerControl
  DV_SPI.c (1314): [WARNING] PowerControl (ARM_POWER_LOW) is not supported
                                          PASSED
...

Test Summary: 56 Tests, 24 Passed, 1 Failed.
Test Result: FAILED
\endverbatim

From previous report it is clear that one test function has failed.<br>
By Inspecting the details in previous report it is clear that <c>TEST 03: SPI_Initialize_Uninitialize</c> has failed
on multiple assertions.<br>
Each failed assertion is recorded as a single  line in the test report.<br>
The failed assert information in the output report contains additional information about the <b>source module</b> and <b>line</b>
in that module where the assertion is located with additional debugging info if available.

The documentation can be consulted regarding the failed function, for example in previous case
documentation on the \ref SPI_Initialize_Uninitialize can be consulted.

Main way of fixing the driver consists of opening reported file mentioned as failed and inspecting the
line in which failure was reported.

If there are many failures, it is recommended to deselect all tests except first failing one
so it is easier to focus on just that failure.
Also, selecting only first failing test removes potential clutter from following failing tests that
are all failing due to same cause.

In the previous report, opening <b>DV_SPI.c</b> module (available in the project) and inspecting the <b>1106</b> line
which states:
\verbatim
  // Driver is uninitialized and peripheral is powered-off:
  // Call PowerControl(ARM_POWER_FULL) function and assert that it returned ARM_DRIVER_ERROR status
  TEST_ASSERT(drv->PowerControl (ARM_POWER_FULL) == ARM_DRIVER_ERROR);
\endverbatim

informs us that call to <c>PowerControl (ARM_POWER_FULL)</c>, when driver is not initialized, is expected to
return <c>ARM_DRIVER_ERROR</c> status code but it has returned a different status code instead.

We should put a breakpoint to this line and start the debug session.<br>
When the breakpoint is hit we can see that a call to <c>PowerControl (ARM_POWER_FULL)</c> has returned <c>ARM_DRIVER_OK</c>
instead of expected <c>ARM_DRIVER_ERROR</c> status code.

We can now go into source code of the driver and fix this.

After we have fixed the driver, the report now looks like below:

\verbatim
CMSIS-Driver SPI Test Report   May  6 2020   11:15:30

TEST 01: SPI_GetVersion
  DV_SPI.c (1023): [INFO] Driver API version 2.3, Driver version 2.15
                                          PASSED
TEST 02: SPI_GetCapabilities              PASSED
TEST 03: SPI_Initialize_Uninitialize      PASSED
TEST 04: SPI_PowerControl
  DV_SPI.c (1314): [WARNING] PowerControl (ARM_POWER_LOW) is not supported
                                          PASSED
...

Test Summary: 56 Tests, 25 Passed, 0 Failed.
Test Result: PASSED
\endverbatim

The fix for the assertion failing in line 1106 has also fixed subsequent assertions
in lines 1109 and 1112 thus the driver now reports no failed tests and reports that all
of the 25 executed tests have passed.

This report could be used as an insurance that the SPI Driver on this device is compliant to the CMSIS-Driver specification.

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page resource_requirements Resource Requirements

\section heap_req Heap Memory
Heap memory is used by the memory allocation functions.<br>
It is usually configured in the <b>startup_\<device\>.s</b> or <b>startup_\<device\>.c</b> 
file located under the \b Device component class but it can also, in some cases, be configured by a <b>linker script</b> instead.

Some interface test functions allocate additional buffers from the heap memory.

The CMSIS-Driver Validation framework does not impose heap requirements because it does not use heap memory.

Each interface test module has specific requirements for the heap memory, default requirements are listed below:

| Interface test module   | Heap memory requirement (in bytes) |
| :---------------------: | :--------------------------------: |
| CAN                     | 128                                |
| Ethernet                | 3028                               |
| SPI                     | 12288                              |
| USART                   | 8192                               |

Interface test modules that are not listed in the previous table do not use heap memory.

The system heap memory size must support the largest heap requirement of any used interface test module.<br>
For example, if SPI driver testing is selected heap memory size should be set to at least 12 kB.

Suggested value for heap memory size is <b>16384</b> bytes.

\note Each module contains additional settings in related configuration file which are not exposed through
      Configuration Wizard and impact the heap memory requirement.<br>
      If these values are changed please adjust heap memory size accordingly.

\section rtos2_req CMSIS-RTOS2

The thread requirements need to be reflected in the CMSIS-RTOS2 configuration. Refer to the
<a class="el" href="https://arm-software.github.io/CMSIS_5/RTOS2/html/index.html" target="_blank">CMSIS-RTOS2 Reference</a> for further details.

For <a class="el" href="https://arm-software.github.io/CMSIS_5/RTOS2/html/rtx5_impl.html" target="_blank">CMSIS-RTOS2 RTX5</a>, thread
requirements are configured in the
<a class=el href="https://arm-software.github.io/CMSIS_5/RTOS2/html/config_rtx5.html" target="_blank">RTX_Config.h</a> file located
under the \b CMSIS component class:

| Option                                                            | Value                              |
| :---------------------------------------------------------------- | :--------------------------------: |
| System Configuration: Global Dynamic Memory size [bytes] (Note 1) | 16384                              |
| Thread Configuration: Default Thread Stack size [bytes]           | 3072                               |

\note Note 1: This setting is only necessary for WiFi driver testing, for other driver testing value of 4096 bytes is sufficient.

*/
/*=======0=========1=========2=========3=========4=========5=========6=========7=========8=========9=========0=========1====*/
/**
\page examples Examples

The CMSIS-Driver Validation Software Pack contains a set of examples that show how to use the CMSIS-Driver Validation on
a real hardware.<br>

Follow these steps to use the validation examples:

1. In the **CMSIS** extension, click **Create a New Solution**.
2. Select the **Target Board** (**NUCLEO-H563ZI** for **Ethernet** example, **B-U585I-IOT02A** for **all other examples**).
3. Under **Templates, Reference Applications, and Examples**, select **Driver Validation**.
4. Choose **Solution Base Folder**, click **Create**.
5. Setup the required hardware as described in the example's **README.md** file
6. Build and Run the application

For details about specific examples check **README.md** of desired example in the example's root folder.

Available examples:

| Validation for Driver  | Board             | Additional required hardware                      |
|------------------------|-------------------|---------------------------------------------------|
| Ethernet               | ST NUCLEO-H563ZI  | Ethernet loopback plug                            |
| GPIO                   | ST B-U585I-IOT02A | ST STM32F429I-DISC1, 1kOhm resistor               |
| SPI                    | ST B-U585I-IOT02A | ST STM32F429I-DISC1, 10kOhm resistor, some wires  |
| USART                  | ST B-U585I-IOT02A | ST STM32F429I-DISC1, some wires                   |
| WiFi on-board EMW3080  | ST B-U585I-IOT02A | ST B-U585I-IOT02A, PC with MS Windows             |
| WiFi ESP8266           | ST B-U585I-IOT02A | SparkFun ESP8266 Shield, PC with MS Windows       |
| WiFi ISM43662          | ST B-U585I-IOT02A | Inventek ISMART43362-E Shield, PC with MS Windows |
| WiFi WizFi360          | ST B-U585I-IOT02A | WIZnet WizFi360-EVB Shield, PC with MS Windows    |

*/