[PATCH 00/21] test: Add support for passing arguments to C unit tests
From: Simon Glass <simon.glass@canonical.com> This series adds infrastructure for passing runtime arguments from Python tests to C unit tests. This makes it easier to support a hybrid testing approach where Python handles complex setup (filesystem images, environment configuration) while C handles the actual test logic with better debuggability. A few other things are included to make this work: - A fix for linker list alignment that was causing garbage values like "Running -858993444 bloblist tests" due to GCC's magic-number division optimization failing when padding breaks exact multiples - A fix fix for serial output with sandbox, since it sometimes misses output at the end when running tests with gnome terminal - Improvements to the linker-list script to detect padding and pointer-arithmetic bugs - A new UNIT_TEST_ARGS() macro for declaring tests with typed arguments, along with argument parsing in the ut command (name=value format) - Argument-accessor macros ut_str(), ut_int(), and ut_bool() with type-checking and bounds validation - A private buffer (uts->priv) for test-local temporary data, which makes it a little easier to write shorter tests - Tests for the argument feature (test_args) covering type checking, bounds checking, and argument-parsing failures As an example, the basic filesystem tests are converted from pure Python to C with Python wrappers. Some improved printf documentation and support for Linux's %pV format are provided. The slight increase in size causes qemu-riscv64_spl to fail, so this series also includes a patch to increase the SPL-malloc() space. Heinrich Schuchardt (1): configs: raise SPL_SYS_MALLOC_SIZE to 8 MiB on RISC-V Simon Glass (20): serial: Retry output if it fails sandbox: serial: Report output failurs doc: Expand printf documentation doc: Document tiny printf for SPL vsprintf: Add support for the %pV format-specifier check_linker_lists: Enhance detection of alignment problems linker_lists: Fix end-marker alignment to prevent padding test: vbe: Fix the ut-flag order in vbe_test_fixup_norun() test: Add a helper to check the next line against a regex test: Add argument-type definitions test: Add a macro to declare unit tests with arguments test: Add support for passing arguments to C tests test: Enhance the ut command to pass test arguments test: Add type-checked argument accessor functions test: Add a private buffer for tests test: Allow preserving console recording on failure test: Add tests for unit-test arguments test: fs: add C-based filesystem tests test: fs: Update Python tests to call C implementations test: Add documentation for test parameters arch/sandbox/cpu/spl.c | 3 +- common/spl/Kconfig | 1 + doc/develop/printf.rst | 174 +++++++++- doc/develop/tests_writing.rst | 58 ++++ doc/usage/cmd/ut.rst | 18 +- drivers/serial/sandbox.c | 7 +- drivers/serial/serial-uclass.c | 2 + include/linker_lists.h | 22 +- include/linux/printk.h | 5 + include/test/fs.h | 39 +++ include/test/test.h | 109 ++++++ include/test/ut.h | 83 ++++- lib/vsprintf.c | 12 + scripts/check_linker_lists.py | 206 ++++++++---- test/Makefile | 1 + test/cmd_ut.c | 38 ++- test/common/Makefile | 1 + test/common/print.c | 48 +++ test/common/test_args.c | 186 +++++++++++ test/fs/Makefile | 3 + test/fs/fs_basic.c | 492 ++++++++++++++++++++++++++++ test/py/tests/test_fs/conftest.py | 4 +- test/py/tests/test_fs/test_basic.py | 346 ++++++------------- test/py/tests/test_vbe.py | 2 +- test/test-main.c | 170 +++++++++- test/ut.c | 94 +++++- 26 files changed, 1776 insertions(+), 348 deletions(-) create mode 100644 include/test/fs.h create mode 100644 test/common/test_args.c create mode 100644 test/fs/Makefile create mode 100644 test/fs/fs_basic.c -- 2.43.0 base-commit: fc71a84bb49e5183eb275a63d7fdbc260c99459d branch: tesf
From: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> On several RISC-V boards we have seen that 1 MiB is a insufficient value for CONFIG_SPL_SYS_MALLOC_SIZE. For instance qemu-riscv32_spl_defconfig fails booting because u-boot.itb exceeds 1 MiB. 8 MiB is a reasonable value that allows adding FPGA blobs or splash images to main U-boot. Reported-by: Simon Glass <sjg@chromium.org> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> Reviewed-by: Leo Yu-Chi Liang <ycliang@andestech.com> (cherry picked from commit 8b410cab51266a0f6ee9c20e7f2bac9cfec079e8) Dropped modifications to all files except common/spl/Kconfig: Signed-off-by: Simon Glass <simon.glass@canonical.com> --- common/spl/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/common/spl/Kconfig b/common/spl/Kconfig index da0a358b2a3..c40dc5a1fe9 100644 --- a/common/spl/Kconfig +++ b/common/spl/Kconfig @@ -481,6 +481,7 @@ config SPL_CUSTOM_SYS_MALLOC_ADDR config SPL_SYS_MALLOC_SIZE hex "Size of the SPL malloc pool" depends on SPL_SYS_MALLOC + default 0x800000 if RISCV default 0x100000 config SPL_READ_ONLY -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> If something goes wrong when writing to the serial device, take notice of the error code and try again, if needed. Signed-off-by: Simon Glass <simon.glass@canonical.com> --- drivers/serial/serial-uclass.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/serial/serial-uclass.c b/drivers/serial/serial-uclass.c index 8644d210770..8d330d687a3 100644 --- a/drivers/serial/serial-uclass.c +++ b/drivers/serial/serial-uclass.c @@ -271,6 +271,8 @@ static int __serial_puts(struct udevice *dev, const char *str, size_t len) do { ssize_t written = ops->puts(dev, str, len); + if (written == -EAGAIN) + continue; if (written < 0) return written; str += written; -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> When characters cannot be sent to the serial port, return an error code so that the uclass can try again. This fixes a problem with gnome-terminal which seems to get behind and then drop output when a large amount of output is produced quickly (e.g. 100K in under a second). Since sandbox often writes to the device one character at a time it does place a substantial load on the other end of the PTY. Quite possibly it does not empty the pipe quickly enough and so writing to the PTY fails. Signed-off-by: Simon Glass <simon.glass@canonical.com> --- drivers/serial/sandbox.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/serial/sandbox.c b/drivers/serial/sandbox.c index cc0491bc3c8..227524ae911 100644 --- a/drivers/serial/sandbox.c +++ b/drivers/serial/sandbox.c @@ -94,13 +94,16 @@ static void sandbox_print_color(struct udevice *dev) static int sandbox_serial_putc(struct udevice *dev, const char ch) { struct sandbox_serial_priv *priv = dev_get_priv(dev); + ssize_t ret; if (ch == '\n') priv->start_of_line = true; if (sandbox_serial_enabled) { sandbox_print_color(dev); - os_write(1, &ch, 1); + ret = os_write(1, &ch, 1); + if (ret != 1) + return -EAGAIN; } _sandbox_serial_written += 1; return 0; @@ -120,6 +123,8 @@ static ssize_t sandbox_serial_puts(struct udevice *dev, const char *s, ret = os_write(1, s, len); if (ret < 0) return ret; + if (ret != len) + return -EAGAIN; } else { ret = len; } -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Expand the printf format documentation to include information on all printf-style functions. Mention CONFIG depencies as well. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/printf.rst | 80 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 70 insertions(+), 10 deletions(-) diff --git a/doc/develop/printf.rst b/doc/develop/printf.rst index 99d05061b14..4a0f054aae1 100644 --- a/doc/develop/printf.rst +++ b/doc/develop/printf.rst @@ -1,7 +1,62 @@ .. SPDX-License-Identifier: GPL-2.0+ -Printf() format codes -===================== +Printf-style Functions +====================== + +U-Boot provides a family of printf-style functions for formatted output. + +Functions +--------- + +printf() + Prints formatted output to the console. + + .. code-block:: c + + int printf(const char *fmt, ...); + +vprintf() + Like printf() but takes a va_list argument. + + .. code-block:: c + + int vprintf(const char *fmt, va_list args); + +sprintf() + Prints formatted output to a string buffer. The buffer must be large + enough to hold the output. + + .. code-block:: c + + int sprintf(char *buf, const char *fmt, ...); + +vsprintf() + Like sprintf() but takes a va_list argument. + + .. code-block:: c + + int vsprintf(char *buf, const char *fmt, va_list args); + +snprintf() + Prints formatted output to a string buffer with a size limit. At most + size-1 characters are written, and the buffer is always null-terminated. + Returns the number of characters that would have been written if the + buffer were large enough. + + .. code-block:: c + + int snprintf(char *buf, size_t size, const char *fmt, ...); + +vsnprintf() + Like snprintf() but takes a va_list argument. + + .. code-block:: c + + int vsnprintf(char *buf, size_t size, const char *fmt, va_list args); + + +Format Specification +-------------------- Each conversion specification consists of: @@ -166,10 +221,15 @@ Pointers * resource_size_t %pD - prints a UEFI device path + prints a UEFI device path (requires CONFIG_EFI_DEVICE_PATH_TO_TEXT) %pi4, %pI4 - prints IPv4 address, e.g. '192.168.0.1' + prints IPv4 address, e.g. '192.168.0.1'. Lower case (%pi4) omits the + dot separators. + +%pi6, %pI6 + prints IPv6 address (requires CONFIG_IPV6). Lower case (%pi6) omits the + colon separators. %pm prints MAC address without separators, e.g. '001122334455' @@ -178,22 +238,22 @@ Pointers print MAC address colon separated, e.g. '00:01:02:03:04:05' %pUb - prints GUID big endian, lower case + prints GUID big endian, lower case (requires CONFIG_LIB_UUID) e.g. '00112233-4455-6677-8899-aabbccddeeff' %pUB - prints GUID big endian, upper case + prints GUID big endian, upper case (requires CONFIG_LIB_UUID) e.g. '00112233-4455-6677-8899-AABBCCDDEEFF' %pUl - prints GUID little endian, lower case + prints GUID little endian, lower case (requires CONFIG_LIB_UUID) e.g. '33221100-5544-7766-8899-aabbccddeeff' %pUL - prints GUID little endian, upper case + prints GUID little endian, upper case (requires CONFIG_LIB_UUID) e.g. '33221100-5544-7766-8899-AABBCCDDEEFF' %pUs prints text description of a GUID or if such is not known little endian, - lower case, e.g. 'system' for a GUID identifying an EFI system - partition. + lower case (requires CONFIG_LIB_UUID), e.g. 'system' for a GUID + identifying an EFI system partition. -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add documentation for the tiny printf implementation used in SPL, TPL, and VPL when CONFIG_SPL_USE_TINY_PRINTF (or equivalent) is enabled. Document the supported format specifiers, limitations, and the warning about snprintf() not performing bounds checking in this implementation. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/printf.rst | 75 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/doc/develop/printf.rst b/doc/develop/printf.rst index 4a0f054aae1..edda8b24a8e 100644 --- a/doc/develop/printf.rst +++ b/doc/develop/printf.rst @@ -257,3 +257,78 @@ Pointers prints text description of a GUID or if such is not known little endian, lower case (requires CONFIG_LIB_UUID), e.g. 'system' for a GUID identifying an EFI system partition. + + +Tiny printf +----------- + +For space-constrained environments like SPL, U-Boot provides a minimal printf +implementation enabled by CONFIG_SPL_USE_TINY_PRINTF (and corresponding +CONFIG_TPL_USE_TINY_PRINTF, CONFIG_VPL_USE_TINY_PRINTF for TPL and VPL). This +reduces code size by approximately 2.5KiB on armv7. + +The tiny printf supports only a limited set of format specifiers: + +Basic specifiers +'''''''''''''''' + +%c + prints a single character + +%s + prints a string + +%d, %i + signed decimal integer + +%u + unsigned decimal integer + +%x + unsigned hexadecimal (lowercase) + +%% + a literal '%' character + +Length modifiers +'''''''''''''''' + +%l + long (e.g., %ld, %lu, %lx) + +Width and padding +''''''''''''''''' + +Field width and zero-padding are supported (e.g., %08x, %4d). + +Pointer specifiers (CONFIG_SPL_NET only) +'''''''''''''''''''''''''''''''''''''''' + +When CONFIG_SPL_NET is enabled, the following pointer formats are available: + +%pM + MAC address colon-separated, e.g. '00:11:22:33:44:55' + +%pm + MAC address without separators, e.g. '001122334455' + +%pI4 + IPv4 address, e.g. '192.168.1.1' + +Limitations +''''''''''' + +The tiny printf does NOT support: + +* Floating point (%f, %e, %g, etc.) +* Long long (%ll) +* Size/ptrdiff modifiers (%z, %t) +* Precision (%.Nf, %.Ns) +* Most pointer formats (%pU, %pD, %pV, etc.) +* The snprintf() size parameter is ignored - no bounds checking is performed + +.. warning:: + + Because snprintf() ignores the size parameter in tiny printf, buffer + overflows are possible. Ensure buffers are large enough for the expected + output. -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add support for the %pV format-specifier which allows printing a struct va_format. This is used by the Linux kernel for recursive printf() formatting and is needed by the ext4l filesystem driver. Add the struct to include/linux/printk.h to match the kernel location. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/printf.rst | 19 +++++++++++++++++ include/linux/printk.h | 5 +++++ lib/vsprintf.c | 12 +++++++++++ test/common/print.c | 48 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 84 insertions(+) diff --git a/doc/develop/printf.rst b/doc/develop/printf.rst index edda8b24a8e..42de22c8555 100644 --- a/doc/develop/printf.rst +++ b/doc/develop/printf.rst @@ -258,6 +258,25 @@ Pointers lower case (requires CONFIG_LIB_UUID), e.g. 'system' for a GUID identifying an EFI system partition. +%pV + prints a struct va_format, which contains a format string and a va_list + pointer. This allows recursive printf formatting and is used for + implementing custom print functions that wrap printf. + + .. code-block:: c + + void my_print(const char *fmt, ...) + { + struct va_format vaf; + va_list args; + + va_start(args, fmt); + vaf.fmt = fmt; + vaf.va = &args; + printf("prefix: %pV\n", &vaf); + va_end(args); + } + Tiny printf ----------- diff --git a/include/linux/printk.h b/include/linux/printk.h index 5e85513853c..e28cef0ac31 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -84,4 +84,9 @@ #define printk_once(fmt, ...) \ printk(fmt, ##__VA_ARGS__) +struct va_format { + const char *fmt; + va_list *va; +}; + #endif diff --git a/lib/vsprintf.c b/lib/vsprintf.c index c072b44140b..0f2c303b138 100644 --- a/lib/vsprintf.c +++ b/lib/vsprintf.c @@ -25,6 +25,7 @@ #include <linux/err.h> #include <linux/types.h> #include <linux/string.h> +#include <linux/printk.h> /* we use this so that we can do without the ctype library */ #define is_digit(c) ((c) >= '0' && (c) <= '9') @@ -508,6 +509,17 @@ static char *pointer(const char *fmt, char *buf, char *end, void *ptr, return uuid_string(buf, end, ptr, field_width, precision, flags, fmt); #endif + case 'V': + { + const struct va_format *vaf = ptr; + va_list va; + + va_copy(va, *vaf->va); + buf += vsnprintf(buf, end > buf ? end - buf : 0, + vaf->fmt, va); + va_end(va); + return buf; + } default: break; } diff --git a/test/common/print.c b/test/common/print.c index 3fe24dc3e9d..9bd409d4b66 100644 --- a/test/common/print.c +++ b/test/common/print.c @@ -9,8 +9,10 @@ #include <log.h> #include <mapmem.h> #include <version_string.h> +#include <stdarg.h> #include <stdio.h> #include <vsprintf.h> +#include <linux/printk.h> #include <test/common.h> #include <test/test.h> #include <test/ut.h> @@ -376,3 +378,49 @@ static int snprint(struct unit_test_state *uts) return 0; } COMMON_TEST(snprint, 0); + +/* Helper function to test %pV format specifier */ +static int print_with_va_format(char *buf, size_t size, const char *fmt, ...) +{ + struct va_format vaf; + va_list args; + int ret; + + va_start(args, fmt); + vaf.fmt = fmt; + vaf.va = &args; + ret = snprintf(buf, size, "prefix: %pV :suffix", &vaf); + va_end(args); + + return ret; +} + +/* Test printing with %pV (struct va_format) */ +static int print_va_format(struct unit_test_state *uts) +{ + char str[64]; + int ret; + + /* Basic string */ + ret = print_with_va_format(str, sizeof(str), "hello"); + ut_asserteq_str("prefix: hello :suffix", str); + ut_asserteq(21, ret); + + /* String with arguments */ + ret = print_with_va_format(str, sizeof(str), "value=%d", 42); + ut_asserteq_str("prefix: value=42 :suffix", str); + ut_asserteq(24, ret); + + /* Multiple arguments */ + ret = print_with_va_format(str, sizeof(str), "%s: %d/%d", "test", 1, 2); + ut_asserteq_str("prefix: test: 1/2 :suffix", str); + ut_asserteq(25, ret); + + /* Truncation */ + ret = print_with_va_format(str, 15, "hello world"); + ut_asserteq_str("prefix: hello ", str); + ut_asserteq(27, ret); + + return 0; +} +COMMON_TEST(print_va_format, 0); -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> When linker-inserted padding breaks list integrity, pointer arithmetic like (end - start) / sizeof(struct) produces garbage. GCC optimizes division by constants using multiplicative inverses, which only works when the dividend is an exact multiple. With padding, outputs like "Running -858993444 bloblist tests" appear instead of the correct count. Enhance the linker list checking script to detect these problems by adding symbol size tracking using nm -S. This enables: 1. Padding detection: Compare symbol sizes to gaps. If gap > size, padding was inserted, breaking contiguous array assumptions. 2. Pointer arithmetic bugs: Check if (end - start) marker span is a multiple of struct size. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- scripts/check_linker_lists.py | 206 +++++++++++++++++++++++----------- 1 file changed, 143 insertions(+), 63 deletions(-) diff --git a/scripts/check_linker_lists.py b/scripts/check_linker_lists.py index c91b1d9875d..4f122480481 100755 --- a/scripts/check_linker_lists.py +++ b/scripts/check_linker_lists.py @@ -1,127 +1,194 @@ #!/usr/bin/env python3 -# -# check_list_alignment.py: Auto-discover and verify the uniform -# spacing of all U-Boot linker list symbols -# -# Analyze the symbol table of a U-Boot ELF file to ensure that -# all entries in all linker-generated lists are separated by a consistent -# number of bytes. Detect problems caused by linker-inserted -# alignment padding. -# -# By default, produce no output if no problems are found -# Use the -v flag to force output even on success -# -# Exit Codes: -# 0: Success. No alignment problems were found -# 1: Usage Error. The script was not called with the correct arguments -# 2: Execution Error. Failed to run `nm` or the ELF file was not found -# 3: Problem Found. An inconsistent gap was detected in at least one list -# +# SPDX-License-Identifier: GPL-2.0+ +"""Check alignment of U-Boot linker lists. + +Auto-discover and verify the uniform spacing of all U-Boot linker list symbols. + +Analyze the symbol table of a U-Boot ELF file to ensure that all entries in all +linker-generated lists are separated by a consistent number of bytes. Detect +problems caused by linker-inserted alignment padding. + +By default, produce no output if no problems are found. +Use the -v flag to force output even on success. + +Exit Codes: + 0: Success - no alignment problems were found + 1: Usage Error - the script was not called with the correct arguments + 2: Execution Error - failed to run `nm` or the ELF file was not found + 3: Problem Found - an inconsistent gap was detected in at least one list +""" import sys import subprocess import re import argparse -from statistics import mode, StatisticsError +from statistics import mode from collections import defaultdict, namedtuple +# Information about a symbol: address, size (from nm -S), and name +Symbol = namedtuple('Symbol', ['address', 'size', 'name']) + # Information about the gap between two consecutive symbols -Gap = namedtuple('Gap', ['gap', 'prev_sym', 'next_sym']) +Gap = namedtuple('Gap', ['gap', 'prev_sym', 'next_sym', 'prev_size']) + +# Start and end marker addresses for a list +Markers = namedtuple('Markers', ['start', 'end']) + # Holds all the analysis results from checking the lists Results = namedtuple('Results', [ 'total_problems', 'total_symbols', 'all_lines', 'max_name_len', 'list_count']) def eprint(*args, **kwargs): - '''Print to stderr''' + """Print to stderr""" print(*args, file=sys.stderr, **kwargs) -def check_single_list(name, symbols, max_name_len): - '''Check alignment for a single list and return its findings +def check_single_list(name, symbols, max_name_len, marker_info=None): + """Check alignment for a single list and return its findings Args: name (str): The cleaned-up name of the list for display - symbols (list): A list of (address, name) tuples, sorted by address + symbols (list): A list of Symbol tuples, sorted by address max_name_len (int): The max length of list names for column formatting + marker_info (Markers): Optional namedtuple with start and end addresses Returns: tuple: (problem_count, list_of_output_lines) - ''' + """ lines = [] if len(symbols) < 2: return 0, [] gaps = [] for i in range(len(symbols) - 1): - addr1, name1 = symbols[i] - addr2, name2 = symbols[i+1] - gaps.append(Gap(gap=addr2 - addr1, prev_sym=name1, next_sym=name2)) + sym1, sym2 = symbols[i], symbols[i+1] + gaps.append(Gap(gap=sym2.address - sym1.address, prev_sym=sym1.name, + next_sym=sym2.name, prev_size=sym1.size)) expected_gap = mode(g.gap for g in gaps) - lines.append( - f"{name:<{max_name_len + 2}} {len(symbols):>12} " - f"{f'0x{expected_gap:x}':>17}") problem_count = 0 + hex_gap = f'0x{expected_gap:x}' + line = f'{name:<{max_name_len + 2}} {len(symbols):>12} {hex_gap:>17}' + lines.append(line) + for g in gaps: if g.gap != expected_gap: problem_count += 1 lines.append( - f" - Bad gap (0x{g.gap:x}) before symbol: {g.next_sym}") + f' - Bad gap (0x{g.gap:x}) before symbol: {g.next_sym}') + elif g.prev_size and g.gap > g.prev_size: + # Gap is larger than symbol size - padding was inserted + problem_count += 1 + lines.append( + f' - Padding: gap 0x{g.gap:x} > size 0x{g.prev_size:x}' + f' before: {g.next_sym}') + + # Check if start/end marker span is a multiple of the struct size + # If not, pointer subtraction (end - start) will produce wrong results + # due to compiler optimization using magic number multiplication + if marker_info: + total_span = marker_info.end - marker_info.start + if total_span % expected_gap != 0: + problem_count += 1 + remainder = total_span % expected_gap + lines.append( + f' - Pointer arithmetic bug: span 0x{total_span:x} is not a ' + f'multiple of struct size 0x{expected_gap:x} ' + f'(remainder: {remainder})') return problem_count, lines def run_nm_and_get_lists(elf_path): - '''Run `nm` and parse the output to discover all linker lists + """Run `nm -S` and parse the output to discover all linker lists Args: elf_path (str): The path to the ELF file to process Returns: - dict or None: A dictionary of discovered lists, or None on error - ''' - cmd = ['nm', '-n', elf_path] + tuple or None: (lists_dict, markers_dict) or None on error + lists_dict: entries keyed by base_name + markers_dict: start/end marker addresses keyed by base_name + """ + cmd = ['nm', '-S', '-n', elf_path] try: proc = subprocess.run(cmd, capture_output=True, text=True, check=True) except FileNotFoundError: eprint( - 'Error: The "nm" command was not found. ' + "Error: The 'nm' command was not found. " 'Please ensure binutils is installed') return None except subprocess.CalledProcessError as e: eprint( f"Error: Failed to execute 'nm' on '{elf_path}'.\n" - f" Return Code: {e.returncode}\n Stderr:\n{e.stderr}") + f' Return Code: {e.returncode}\n Stderr:\n{e.stderr}') return None - list_name_pattern = re.compile( + # Pattern to match _2_ entries (the actual list elements) + entry_pattern = re.compile( r'^(?P<base_name>_u_boot_list_\d+_\w+)(?:_info)?_2_') + # Pattern to match _1 (start) and _3 (end) markers + marker_pattern = re.compile( + r'^(?P<base_name>_u_boot_list_\d+_\w+)_(?P<marker>[13])$') + lists = defaultdict(list) + markers = defaultdict(dict) # {base_name: {'start': addr, 'end': addr}} + for line in proc.stdout.splitlines(): - if ' D _u_boot_list_' not in line: + if '_u_boot_list_' not in line: continue try: parts = line.strip().split() - address, name = int(parts[0], 16), parts[-1] - - match = list_name_pattern.match(name) + name = parts[-1] + address = int(parts[0], 16) + # Size is present if we have 4 parts and parts[2] is a single char + if len(parts) == 4 and len(parts[2]) == 1: + size = int(parts[1], 16) + else: + size = 0 # Size not available + + # Check for entry (_2_) symbols - must be uppercase D + if ' D _u_boot_list_' in line: + match = entry_pattern.match(name) + if match: + base_name = match.group('base_name') + lists[base_name].append(Symbol(address, size, name)) + continue + + # Check for marker (_1 or _3) symbols - can be any type + match = marker_pattern.match(name) if match: base_name = match.group('base_name') - lists[base_name].append((address, name)) + marker_type = match.group('marker') + if marker_type == '1': + markers[base_name]['start'] = address + else: # marker_type == '3' + markers[base_name]['end'] = address + except (ValueError, IndexError): eprint(f'Warning: Could not parse line: {line}') - return lists + # Convert marker dicts to Markers namedtuples (only if both start/end exist) + marker_tuples = {} + for base_name, m in markers.items(): + if 'start' in m and 'end' in m: + marker_tuples[base_name] = Markers(m['start'], m['end']) -def collect_data(lists): - '''Collect alignment check data for all lists + return lists, marker_tuples + +def collect_data(lists, markers): + """Collect alignment check data for all lists Args: lists (dict): A dictionary of lists and their symbols + markers (dict): A dictionary of start/end marker addresses per list Returns: Results: A namedtuple containing the analysis results - ''' + """ + if markers is None: + markers = {} + names = {} prefix_to_strip = '_u_boot_list_2_' for list_name in lists.keys(): @@ -138,7 +205,9 @@ def collect_data(lists): symbols = lists[list_name] total_symbols += len(symbols) name = names[list_name] - problem_count, lines = check_single_list(name, symbols, max_name_len) + marker_info = markers.get(list_name) + problem_count, lines = check_single_list(name, symbols, max_name_len, + marker_info) total_problems += problem_count all_lines.extend(lines) @@ -150,19 +219,20 @@ def collect_data(lists): list_count=len(lists)) def show_output(results, verbose): - '''Print the collected results to stderr based on verbosity + """Print the collected results to stderr based on verbosity Args: results (Results): The analysis results from collect_data() verbose (bool): True to print output even on success - ''' + """ if results.total_problems == 0 and not verbose: return header = (f"{'List Name':<{results.max_name_len + 2}} {'# Symbols':>12} " f"{'Struct Size (hex)':>17}") + sep = f"{'-' * (results.max_name_len + 2)} {'-' * 12} {'-' * 17}" eprint(header) - eprint(f"{'-' * (results.max_name_len + 2)} {'-' * 12} {'-' * 17}") + eprint(sep) for line in results.all_lines: eprint(line) @@ -177,19 +247,28 @@ def show_output(results, verbose): eprint('\nSUCCESS: All discovered lists have consistent alignment') def main(): - '''Main entry point of the script, returns an exit code''' + """Main entry point of the script, returns an exit code""" epilog_text = ''' Auto-discover all linker-generated lists in a U-Boot ELF file -(e.g., for drivers, commands, etc.) and verify their integrity. Check -that all elements in a given list are separated by a consistent number of -bytes. +(e.g., for drivers, commands, etc.) and verify their integrity. + +Problems detected (cause build failure): + +1. Inconsistent gaps: Elements in a list should all be separated by the same + number of bytes (the struct size). If the linker inserts padding between + some elements but not others, this is detected and reported. + +2. Padding detection: Using symbol sizes from nm -S, the script compares each + symbol's size to the gap after it. If gap > size, the linker inserted + padding, which breaks U-Boot's assumption that the list is a contiguous + array of same-sized structs. -Problems typically indicate that the linker has inserted alignment padding -between two elements in a list, which can break U-Boot's assumption that the -list is a simple, contiguous array of same-sized structs. +3. Pointer arithmetic bugs: Each list has start (_1) and end (_3) markers. + If the span (end - start) is not a multiple of struct size, pointer + subtraction produces garbage due to GCC's magic-number division. ''' parser = argparse.ArgumentParser( - description='Check alignment of all U-Boot linker lists in an ELF file.', + description='Check alignment of U-Boot linker lists in an ELF file.', epilog=epilog_text, formatter_class=argparse.RawDescriptionHelpFormatter ) @@ -200,16 +279,17 @@ list is a simple, contiguous array of same-sized structs. args = parser.parse_args() - lists = run_nm_and_get_lists(args.elf_path) - if lists is None: + result = run_nm_and_get_lists(args.elf_path) + if result is None: return 2 # Error running nm + lists, markers = result if not lists: if args.verbose: eprint('Success: No U-Boot linker lists found to check') return 0 - results = collect_data(lists) + results = collect_data(lists, markers) show_output(results, args.verbose) return 3 if results.total_problems > 0 else 0 -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Change the alignment of end markers in ll_entry_end() and ll_end_decl() from __aligned(4) and __aligned(CONFIG_LINKER_LIST_ALIGN) respectively to __aligned(1). The linker places zero-size end markers at aligned boundaries based on what follows them. When the next list's start marker has a high alignment requirement (e.g., 32 bytes), padding gets inserted before the end marker. This causes the byte span (end - start) to not be an exact multiple of the struct size. The compiler optimizes pointer subtraction (end - start) using magic-number multiplication for division. This optimization only produces correct results when the byte span is an exact multiple of the struct size. With padding, the result is garbage (e.g., -858993444 instead of 15). By using __aligned(1), the end marker is placed immediately after the last entry with no padding, ensuring (end - start) equals exactly n * sizeof where n is the number of entries. This makes ll_entry_count() and direct pointer arithmetic work correctly. Fixes: 0b2fa98aa5e5 ("linker_lists: Fix alignment issue") Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/linker_lists.h | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/include/linker_lists.h b/include/linker_lists.h index 0f4a2d686e2..6a018f175ca 100644 --- a/include/linker_lists.h +++ b/include/linker_lists.h @@ -145,6 +145,20 @@ * Since this macro defines an array end symbol, its leftmost index * must be 2 and its rightmost index must be 3. * + * The end symbol uses __aligned(1) to ensure it is placed immediately after + * the last entry without any padding. This is critical for ll_entry_count() + * to work correctly. + * + * If the end marker had a higher alignment (e.g., 4 or 32 bytes), the linker + * might insert padding between the last entry and the end marker to satisfy + * alignment requirements of the following section. This would cause pointer + * subtraction (end - start) to produce incorrect results because the compiler + * optimizes pointer division using magic-number multiplication, which only + * works correctly when the byte span is an exact multiple of the struct size. + * + * With __aligned(1), the end marker is placed at exactly (start + n * sizeof) + * where n is the number of entries, ensuring correct pointer arithmetic. + * * Example: * * :: @@ -153,7 +167,7 @@ */ #define ll_entry_end(_type, _list) \ ({ \ - static char end[0] __aligned(4) __attribute__((unused)) \ + static char end[0] __aligned(1) __attribute__((unused)) \ __section("__u_boot_list_2_"#_list"_3"); \ _type * tmp = (_type *)&end; \ asm("":"+r"(tmp)); \ @@ -239,8 +253,12 @@ static _type _sym[0] __aligned(CONFIG_LINKER_LIST_ALIGN) \ __maybe_unused __section("__u_boot_list_2_" #_list "_1") +/* + * ll_end_decl uses __aligned(1) to avoid padding before the end marker. + * See the comment for ll_entry_end() for a full explanation. + */ #define ll_end_decl(_sym, _type, _list) \ - static _type _sym[0] __aligned(CONFIG_LINKER_LIST_ALIGN) \ + static _type _sym[0] __aligned(1) \ __maybe_unused __section("__u_boot_list_2_" #_list "_3") /** -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Move the -f flag before the suite name since ut parses flags before the suite argument. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- test/py/tests/test_vbe.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/py/tests/test_vbe.py b/test/py/tests/test_vbe.py index e5b489bc819..4ccf4fb937b 100644 --- a/test/py/tests/test_vbe.py +++ b/test/py/tests/test_vbe.py @@ -87,7 +87,7 @@ bootm loados bootm prep fdt addr fdt print -ut bootstd -f vbe_test_fixup_norun +ut -f bootstd vbe_test_fixup_norun ''' @pytest.mark.boardspec('sandbox') -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add a new ut_assert_nextline_regex() macro and ut_check_console_line_regex() helper to check console output against a regex pattern. This is useful when the exact output varies (e.g., file paths or line numbers in error messages). Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/tests_writing.rst | 3 +++ include/test/ut.h | 29 +++++++++++++++++++++++++++++ test/ut.c | 21 +++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index 43756989d43..41612a9e21b 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -451,6 +451,9 @@ ut_assert_nextlinen(fmt, args...) Assert that the next console output line matches up to the format string length +ut_assert_nextline_regex(pattern) + Assert that the next console output line matches a regex pattern + ut_assert_nextline_empty() Assert that the next console output line is empty diff --git a/include/test/ut.h b/include/test/ut.h index a2b42cdf414..7098c9be7d6 100644 --- a/include/test/ut.h +++ b/include/test/ut.h @@ -87,6 +87,20 @@ int ut_check_console_line(struct unit_test_state *uts, const char *fmt, ...) int ut_check_console_linen(struct unit_test_state *uts, const char *fmt, ...) __attribute__ ((format (__printf__, 2, 3))); +/** + * ut_check_console_line_regex() - Check the next console line against a regex + * + * This checks the next line of console output against a regex pattern. + * + * After the function returns, uts->expect_str holds the regex pattern and + * uts->actual_str holds the actual string read from the console. + * + * @uts: Test state + * @regex: Regular expression pattern to match against + * Return: 0 if OK, other value on error + */ +int ut_check_console_line_regex(struct unit_test_state *uts, const char *regex); + /** * ut_check_skipline() - Check that the next console line exists and skip it * @@ -412,6 +426,21 @@ int ut_check_console_dump(struct unit_test_state *uts, int total_bytes); __ret; \ }) +/* Assert that the next console output line matches a regex pattern */ +#define ut_assert_nextline_regex(pattern) ({ \ + int __ret = 0; \ + \ + if (ut_check_console_line_regex(uts, pattern)) { \ + ut_failf(uts, __FILE__, __LINE__, __func__, \ + "console regex", \ + "\nExpected regex '%s',\n got '%s'", \ + uts->expect_str, uts->actual_str); \ + if (!uts->soft_fail) \ + return CMD_RET_FAILURE; \ + } \ + __ret; \ +}) + /* Assert that there is a 'next' console output line, and skip it */ #define ut_assert_skipline() ({ \ int __ret = 0; \ diff --git a/test/ut.c b/test/ut.c index 94b09364687..aed59cae0b9 100644 --- a/test/ut.c +++ b/test/ut.c @@ -150,6 +150,27 @@ int ut_check_console_linen(struct unit_test_state *uts, const char *fmt, ...) strlen(uts->expect_str)); } +int ut_check_console_line_regex(struct unit_test_state *uts, const char *regex) +{ + char err[UT_REGEX_ERR_SIZE]; + int len; + int ret; + + len = strlcpy(uts->expect_str, regex, sizeof(uts->expect_str)); + if (len >= sizeof(uts->expect_str)) { + ut_fail(uts, __FILE__, __LINE__, __func__, + "unit_test_state->expect_str too small"); + return -EOVERFLOW; + } + ret = readline_check(uts); + if (ret == -ENOENT) + return 1; + + ret = ut_check_regex(regex, uts->actual_str, err); + + return ret; +} + int ut_check_skipline(struct unit_test_state *uts) { int ret; -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add types for declaring and storing unit test arguments: - enum ut_arg_type: INT, BOOL, STR types - enum ut_arg_flags: OPTIONAL flag for non-required args - struct ut_arg_def: declares expected args with defaults - struct ut_arg: holds parsed argument values This prepares for passing key=value arguments to tests via the 'ut' command instead of needing to use environment variables. Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/test/test.h | 65 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/include/test/test.h b/include/test/test.h index 086fff1ca26..5ead1200a40 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -27,6 +27,40 @@ struct ut_stats { ulong duration_ms; }; +/** + * enum ut_arg_type - Type of a unit test argument + * + * @UT_ARG_INT: Integer argument (hex with 0x prefix, or decimal) -> vint + * @UT_ARG_BOOL: Boolean argument (0 or 1) -> vbool + * @UT_ARG_STR: String argument -> vstr + */ +enum ut_arg_type { + UT_ARG_INT, + UT_ARG_BOOL, + UT_ARG_STR, +}; + +/** + * struct ut_arg - Parsed unit test argument value + * + * Holds the parsed value of an argument after command-line processing. + * + * @name: Name of the argument (points to ut_arg_def.name) + * @type: Type of the argument + * @vint: Integer value (when type is UT_ARG_INT) + * @vbool: Boolean value (when type is UT_ARG_BOOL) + * @vstr: String value (when type is UT_ARG_STR, points into argv) + */ +struct ut_arg { + const char *name; + enum ut_arg_type type; + union { + long vint; + bool vbool; + const char *vstr; + }; +}; + /* * struct unit_test_state - Entire state of test system * @@ -107,6 +141,37 @@ enum ut_flags { UTF_UNINIT = BIT(14), /* test uninits a suite */ }; +/** + * enum ut_arg_flags - Flags for unit test arguments + * + * @UT_ARGF_OPTIONAL: Argument is optional; use default value if not provided + */ +enum ut_arg_flags { + UT_ARGF_OPTIONAL = BIT(0), +}; + +/** + * struct ut_arg_def - Definition of a unit test argument + * + * Declares an expected argument for a test, including its name, type, + * whether it is optional, and its default value. + * + * @name: Name of the argument (used in key=value matching) + * @type: Type of the argument (int, bool, or string) + * @flags: Argument flags (e.g., UT_ARGF_OPTIONAL) + * @def: Default value (used when argument is optional and not provided) + */ +struct ut_arg_def { + const char *name; + enum ut_arg_type type; + int flags; + union { + long vint; + bool vbool; + const char *vstr; + } def; +}; + /** * struct unit_test - Information about a unit test * -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add a UNIT_TEST_ARGS() macro to declare and register a test with inline argument definitions. The variadic arguments use struct ut_arg_def and a NULL terminator is added automatically to the list. Example usage: static int my_test(struct unit_test_state *uts) { ... } UNIT_TEST_ARGS(my_test, UTF_CONSOLE, my_suite, { "path", UT_ARG_STR }, { "count", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 10 } }); Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/test/test.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/include/test/test.h b/include/test/test.h index 5ead1200a40..cb0539e3bdd 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -47,6 +47,7 @@ enum ut_arg_type { * * @name: Name of the argument (points to ut_arg_def.name) * @type: Type of the argument + * @provided: true if value was provided on command line * @vint: Integer value (when type is UT_ARG_INT) * @vbool: Boolean value (when type is UT_ARG_BOOL) * @vstr: String value (when type is UT_ARG_STR, points into argv) @@ -54,6 +55,7 @@ enum ut_arg_type { struct ut_arg { const char *name; enum ut_arg_type type; + bool provided; union { long vint; bool vbool; @@ -178,12 +180,14 @@ struct ut_arg_def { * @name: Name of test * @func: Function to call to perform test * @flags: Flags indicated pre-conditions for test + * @arg_defs: Argument definitions (NULL-terminated array), or NULL */ struct unit_test { const char *file; const char *name; int (*func)(struct unit_test_state *state); int flags; + const struct ut_arg_def *arg_defs; }; /** @@ -235,6 +239,33 @@ struct unit_test { .func = _name, \ } +/** + * UNIT_TEST_ARGS() - create unit test entry with inline argument definitions + * + * Like UNIT_TEST() but allows specifying argument definitions inline. + * The variadic arguments are struct ut_arg_def initializers. The NULL + * terminator is added automatically by the macro. + * + * Example: + * UNIT_TEST_ARGS(my_test, UTF_CONSOLE, my_suite, + * { "path", UT_ARG_STR }, + * { "count", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 10 } }); + * + * @_name: Test function name + * @_flags: Test flags (see enum ut_flags) + * @_suite: Test suite name + * @...: Argument definitions (struct ut_arg_def initializers) + */ +#define UNIT_TEST_ARGS(_name, _flags, _suite, ...) \ + static const struct ut_arg_def _name##_args[] = { __VA_ARGS__, { NULL } }; \ + ll_entry_declare(struct unit_test, _name, ut_ ## _suite) = { \ + .file = __FILE__, \ + .name = #_name, \ + .flags = _flags, \ + .func = _name, \ + .arg_defs = _name##_args, \ + } + /* Get the start of a list of unit tests for a particular suite */ #define UNIT_TEST_SUITE_START(_suite) \ ll_entry_start(struct unit_test, ut_ ## _suite) -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add support for passing key=value arguments to unit tests. The test framework parses arguments based on definitions provided by each test and makes them available via uts->args[] For now the 'ut' command does not support this new feature. Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- arch/sandbox/cpu/spl.c | 3 +- include/test/test.h | 6 ++ include/test/ut.h | 4 +- test/cmd_ut.c | 3 +- test/test-main.c | 169 ++++++++++++++++++++++++++++++++++++++++- 5 files changed, 178 insertions(+), 7 deletions(-) diff --git a/arch/sandbox/cpu/spl.c b/arch/sandbox/cpu/spl.c index 7ee4975523e..162b678314e 100644 --- a/arch/sandbox/cpu/spl.c +++ b/arch/sandbox/cpu/spl.c @@ -152,7 +152,8 @@ void spl_board_init(void) ut_init_state(&uts); ret = ut_run_list(&uts, "spl", NULL, tests, count, - state->select_unittests, 1, false, NULL); + state->select_unittests, 1, false, NULL, + 0, NULL); ut_report(&uts.cur, 1); ut_uninit_state(&uts); /* continue execution into U-Boot */ diff --git a/include/test/test.h b/include/test/test.h index cb0539e3bdd..da38b8ee4f0 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -9,6 +9,8 @@ #include <malloc.h> #include <linux/bitops.h> +#define UT_MAX_ARGS 8 + /** * struct ut_stats - Statistics about tests run * @@ -90,6 +92,8 @@ struct ut_arg { * @soft_fail: continue execution of the test even after it fails * @expect_str: Temporary string used to hold expected string value * @actual_str: Temporary string used to hold actual string value + * @args: Parsed argument values for current test + * @arg_count: Number of parsed arguments */ struct unit_test_state { struct ut_stats cur; @@ -116,6 +120,8 @@ struct unit_test_state { bool soft_fail; char expect_str[1024]; char actual_str[1024]; + struct ut_arg args[UT_MAX_ARGS]; + int arg_count; }; /* Test flags for each test */ diff --git a/include/test/ut.h b/include/test/ut.h index 7098c9be7d6..90b9bf79929 100644 --- a/include/test/ut.h +++ b/include/test/ut.h @@ -618,12 +618,14 @@ void ut_uninit_state(struct unit_test_state *uts); * name is the name of the test to run. This is used to find which test causes * another test to fail. If the one test fails, testing stops immediately. * Pass NULL to disable this + * @argc: Number of test arguments (key=value pairs), 0 if none + * @argv: Test argument array, NULL if none * Return: 0 if all tests passed, -1 if any failed */ int ut_run_list(struct unit_test_state *uts, const char *category, const char *prefix, struct unit_test *tests, int count, const char *select_name, int runs_per_test, bool force_run, - const char *test_insert); + const char *test_insert, int argc, char *const argv[]); /** * ut_report() - Report stats on a test run diff --git a/test/cmd_ut.c b/test/cmd_ut.c index d8b3c325c41..bad123a75fd 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -132,7 +132,8 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste, snprintf(prefix, sizeof(prefix), "%s_test_", ste->name); ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents, - select_name, runs_per_test, force_run, test_insert); + select_name, runs_per_test, force_run, test_insert, + 0, NULL); return ret; } diff --git a/test/test-main.c b/test/test-main.c index 941b883e156..ac5680d77d0 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -89,6 +89,154 @@ void ut_uninit_state(struct unit_test_state *uts) } } +/** + * ut_count_args() - Count the number of arguments in a NULL-terminated array + * + * @defs: Argument definitions array (NULL-terminated) + * Return: Number of arguments + */ +static int ut_count_args(const struct ut_arg_def *defs) +{ + int count = 0; + + if (defs) { + while (defs[count].name) + count++; + } + + return count; +} + +/** + * ut_set_arg() - Find and set an argument value + * + * Search through argument definitions to find a matching key and set its value. + * + * @defs: Argument definitions array + * @args: Argument values array to update + * @count: Number of argument definitions + * @key: Key name to search for + * @key_len: Length of key name + * @val: Value string to parse + * Return: true if argument was found and set, false otherwise + */ +static bool ut_set_arg(const struct ut_arg_def *defs, struct ut_arg *args, + int count, const char *key, int key_len, const char *val) +{ + int j; + + for (j = 0; j < count; j++) { + if (strlen(defs[j].name) == key_len && + !strncmp(defs[j].name, key, key_len)) { + switch (defs[j].type) { + case UT_ARG_INT: + args[j].vint = simple_strtol(val, NULL, 0); + break; + case UT_ARG_BOOL: + args[j].vbool = *val == '1'; + break; + case UT_ARG_STR: + args[j].vstr = val; + break; + } + args[j].provided = true; + return true; + } + } + + return false; +} + +/** + * ut_parse_args() - Parse command-line arguments for a test + * + * Parse key=value arguments from the command line and set up uts->args based on + * the test's argument definitions. + * + * @uts: Unit test state (args and arg_count will be set) + * @test: Test being run (provides arg_defs) + * @argc: Number of arguments + * @argv: Argument array (key=value strings) + * Return: 0 on success, -EINVAL on parse error + */ +static int ut_parse_args(struct unit_test_state *uts, struct unit_test *test, + int argc, char *const argv[]) +{ + const struct ut_arg_def *defs = test->arg_defs; + struct ut_arg *args = uts->args; + int count = ut_count_args(defs); + int i; + + uts->arg_count = 0; + + /* No arguments expected */ + if (!count) { + if (argc > 0) { + printf("Test '%s' does not accept arguments\n", + test->name); + return -EINVAL; + } + return 0; + } + + if (count > UT_MAX_ARGS) { + printf("Test '%s' has too many arguments (%d > %d)\n", + test->name, count, UT_MAX_ARGS); + return -EINVAL; + } + + /* Initialise from defaults */ + for (i = 0; i < count; i++) { + args[i].name = defs[i].name; + args[i].type = defs[i].type; + args[i].provided = false; + switch (defs[i].type) { + case UT_ARG_INT: + args[i].vint = defs[i].def.vint; + break; + case UT_ARG_BOOL: + args[i].vbool = defs[i].def.vbool; + break; + case UT_ARG_STR: + args[i].vstr = defs[i].def.vstr; + break; + } + } + + /* Parse command-line key=value pairs */ + for (i = 0; i < argc; i++) { + const char *arg = argv[i]; + const char *eq = strchr(arg, '='); + int key_len; + + if (!eq) { + printf("Invalid argument '%s' (expected key=value)\n", + arg); + return -EINVAL; + } + key_len = eq - arg; + + if (!ut_set_arg(defs, args, count, arg, key_len, eq + 1)) { + printf("Unknown argument '%.*s' for test '%s'\n", + key_len, arg, test->name); + return -EINVAL; + } + } + + /* Check required arguments are provided */ + for (i = 0; i < count; i++) { + if (!args[i].provided && !(defs[i].flags & UT_ARGF_OPTIONAL)) { + printf("Missing required argument '%s' for test '%s'\n", + defs[i].name, test->name); + return -EINVAL; + } + } + + uts->arg_count = count; + + return 0; +} + /** * dm_test_pre_run() - Get ready to run a driver model test * @@ -593,12 +741,15 @@ static int ut_run_test_live_flat(struct unit_test_state *uts, * @test_insert: String describing a test to run after n other tests run, in the * format n:name where n is the number of tests to run before this one and * name is the name of the test to run + * @argc: Number of test arguments (key=value pairs) + * @argv: Test argument array * Return: 0 if all tests passed, -ENOENT if test @select_name was not found, * -EBADF if any failed */ static int ut_run_tests(struct unit_test_state *uts, const char *prefix, struct unit_test *tests, int count, - const char *select_name, const char *test_insert) + const char *select_name, const char *test_insert, + int argc, char *const argv[]) { int prefix_len = prefix ? strlen(prefix) : 0; struct unit_test *test, *one; @@ -654,6 +805,11 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix, uts->cur.test_count++; if (one && upto == pos) { + ret = ut_parse_args(uts, one, argc, argv); + if (ret) { + uts->cur.fail_count++; + return ret; + } ret = ut_run_test_live_flat(uts, one, NULL); if (uts->cur.fail_count != old_fail_count) { printf("Test '%s' failed %d times (position %d)\n", @@ -667,6 +823,12 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix, if (prefix_len && !strncmp(test_name, prefix, prefix_len)) test_name = test_name + prefix_len; + ret = ut_parse_args(uts, test, argc, argv); + if (ret) { + found++; + uts->cur.fail_count++; + continue; + } for (i = 0; i < uts->runs_per_test; i++) ret = ut_run_test_live_flat(uts, test, test_name); if (uts->cur.fail_count != old_fail_count) { @@ -706,9 +868,8 @@ void ut_report(struct ut_stats *stats, int run_count) int ut_run_list(struct unit_test_state *uts, const char *category, const char *prefix, struct unit_test *tests, int count, const char *select_name, int runs_per_test, bool force_run, - const char *test_insert) + const char *test_insert, int argc, char *const argv[]) { - ; bool was_bypassed, has_dm_tests = false; ulong start_offset = 0; ulong test_offset = 0; @@ -751,7 +912,7 @@ int ut_run_list(struct unit_test_state *uts, const char *category, uts->force_run = force_run; was_bypassed = pager_set_test_bypass(gd_pager(), true); ret = ut_run_tests(uts, prefix, tests, count, select_name, - test_insert); + test_insert, argc, argv); pager_set_test_bypass(gd_pager(), was_bypassed); /* Best efforts only...ignore errors */ -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Update the ut command to permit passing arguments to tests. Usage: ut -f fs test_name key1=value1 key2=value2 Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/usage/cmd/ut.rst | 11 ++++++++++- test/cmd_ut.c | 29 ++++++++++++++++++++--------- 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index 1acf3126680..d8c3cbf496c 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-r<runs>] [-f] [-I<n>:<one_test>] [-r<n>] [<suite> | 'all' [<test>]] + ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...] ut [-s] info Description @@ -37,6 +37,11 @@ test causes another test to fail. If the one test fails, testing stops immediately. +args + Optional arguments to pass to the test, in `name=value` format. These are + used by tests declared with `UNIT_TEST_ARGS()` which define expected + argument names and types. + Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it includes a near-complete set of emulators, no code-size limits, many CONFIG options enabled and runs easily in CI without needing QEMU. It is also possible @@ -201,3 +206,7 @@ Run a selection of three suites:: Tests run: 10, 12 ms, average: 1 ms, failures: 0 Suites run: 3, total tests run: 37, 26 ms, average: 0 ms, failures: 0 Average test time: 0 ms, worst case 'mem' took 1 ms + +Run a test with arguments (used by tests declared with UNIT_TEST_ARGS):: + + => ut -f fs fs_test_ls_norun fs_type=ext4 fs_image=/tmp/img small=1MB.file diff --git a/test/cmd_ut.c b/test/cmd_ut.c index bad123a75fd..adc96fcbcdc 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -30,7 +30,7 @@ struct suite { static int do_ut_all(struct unit_test_state *uts, const char *select_name, int runs_per_test, bool force_run, - const char *test_insert); + const char *test_insert, int argc, char *const argv[]); static int do_ut_info(bool show_suites); @@ -122,7 +122,7 @@ static bool has_tests(struct suite *ste) /** run_suite() - Run a suite of tests */ static int run_suite(struct unit_test_state *uts, struct suite *ste, const char *select_name, int runs_per_test, bool force_run, - const char *test_insert) + const char *test_insert, int argc, char *const argv[]) { int n_ents = ste->end - ste->start; char prefix[30]; @@ -133,7 +133,7 @@ static int run_suite(struct unit_test_state *uts, struct suite *ste, ret = ut_run_list(uts, ste->name, prefix, ste->start, n_ents, select_name, runs_per_test, force_run, test_insert, - 0, NULL); + argc, argv); return ret; } @@ -169,7 +169,8 @@ static void update_stats(struct unit_test_state *uts, const struct suite *ste) } static int do_ut_all(struct unit_test_state *uts, const char *select_name, - int runs_per_test, bool force_run, const char *test_insert) + int runs_per_test, bool force_run, const char *test_insert, + int argc, char *const argv[]) { int i; int retval; @@ -181,7 +182,7 @@ static int do_ut_all(struct unit_test_state *uts, const char *select_name, if (has_tests(ste)) { printf("----Running %s tests----\n", ste->name); retval = run_suite(uts, ste, select_name, runs_per_test, - force_run, test_insert); + force_run, test_insert, argc, argv); if (!any_fail) any_fail = retval; update_stats(uts, ste); @@ -246,6 +247,8 @@ static struct suite *find_suite(const char *name) static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) { const char *test_insert = NULL, *select_name; + int test_argc; + char *const *test_argv; struct unit_test_state uts; bool show_suites = false; bool force_run = false; @@ -287,9 +290,14 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) ut_init_state(&uts); name = argv[0]; select_name = cmd_arg1(argc, argv); + + /* Test arguments are after suite name and test name */ + test_argc = argc > 2 ? argc - 2 : 0; + test_argv = argc > 2 ? argv + 2 : NULL; + if (!strcmp(name, "all")) { ret = do_ut_all(&uts, select_name, runs_per_text, force_run, - test_insert); + test_insert, test_argc, test_argv); } else if (!strcmp(name, "info")) { ret = do_ut_info(show_suites); } else { @@ -308,7 +316,8 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) } ret = run_suite(&uts, ste, select_name, runs_per_text, - force_run, test_insert); + force_run, test_insert, test_argc, + test_argv); if (!any_fail) any_fail = ret; update_stats(&uts, ste); @@ -324,12 +333,14 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) } U_BOOT_LONGHELP(ut, - "[-rs] [-f] [-I<n>:<one_test>][<suites>] - run unit tests\n" + "[-rs] [-f] [-I<n>:<one_test>] <suite> [<test> [<args>...]] - run unit tests\n" " -r<runs> Number of times to run each test\n" " -f Force 'manual' tests to run as well\n" " -I Test to run after <n> other tests have run\n" " -s Show all suites with ut info\n" - " <suites> Comma-separated list of suites to run\n" + " <suite> Test suite to run (or comma-separated list)\n" + " <test> Specific test to run (optional)\n" + " <args> Test arguments as key=value pairs (optional)\n" "\n" "Options for <suite>:\n" "all - execute all enabled tests\n" -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add ut_get_str(), ut_get_int(), and ut_get_bool() functions with corresponding ut_str(), ut_int(), and ut_bool() macros for accessing test arguments with type checking. These functions check that the argument index is within bounds and the type matches what was requested. The first failure for a test is reported via ut_failf() which should make it fairly easy to debug the test. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/test/test.h | 2 ++ include/test/ut.h | 50 ++++++++++++++++++++++++++++++++++ test/test-main.c | 1 + test/ut.c | 66 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 119 insertions(+) diff --git a/include/test/test.h b/include/test/test.h index da38b8ee4f0..15ed8b37890 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -94,6 +94,7 @@ struct ut_arg { * @actual_str: Temporary string used to hold actual string value * @args: Parsed argument values for current test * @arg_count: Number of parsed arguments + * @arg_error: Set if ut_str/int/bool() detects a type mismatch */ struct unit_test_state { struct ut_stats cur; @@ -122,6 +123,7 @@ struct unit_test_state { char actual_str[1024]; struct ut_arg args[UT_MAX_ARGS]; int arg_count; + bool arg_error; }; /* Test flags for each test */ diff --git a/include/test/ut.h b/include/test/ut.h index 90b9bf79929..4a9a401cf48 100644 --- a/include/test/ut.h +++ b/include/test/ut.h @@ -635,4 +635,54 @@ int ut_run_list(struct unit_test_state *uts, const char *category, */ void ut_report(struct ut_stats *stats, int run_count); +/** + * ut_get_str() - Get a string test argument + * + * Fails the test if the argument type is not UT_ARG_STR. + * + * @uts: Test state + * @n: Argument index + * @file: Filename of caller + * @line: Line number of caller + * @func: Function name of caller + * Return: String value, or NULL if type mismatch + */ +const char *ut_get_str(struct unit_test_state *uts, int n, const char *file, + int line, const char *func); + +/** + * ut_get_int() - Get an integer test argument + * + * Fails the test if the argument type is not UT_ARG_INT. + * + * @uts: Test state + * @n: Argument index + * @file: Filename of caller + * @line: Line number of caller + * @func: Function name of caller + * Return: Integer value, or 0 if type mismatch + */ +long ut_get_int(struct unit_test_state *uts, int n, const char *file, + int line, const char *func); + +/** + * ut_get_bool() - Get a boolean test argument + * + * Fails the test if the argument type is not UT_ARG_BOOL. + * + * @uts: Test state + * @n: Argument index + * @file: Filename of caller + * @line: Line number of caller + * @func: Function name of caller + * Return: Boolean value, or false if type mismatch + */ +bool ut_get_bool(struct unit_test_state *uts, int n, const char *file, + int line, const char *func); + +/* Helpers for accessing test arguments with type checking */ +#define ut_str(n) ut_get_str(uts, n, __FILE__, __LINE__, __func__) +#define ut_int(n) ut_get_int(uts, n, __FILE__, __LINE__, __func__) +#define ut_bool(n) ut_get_bool(uts, n, __FILE__, __LINE__, __func__) + #endif diff --git a/test/test-main.c b/test/test-main.c index ac5680d77d0..c9e164da678 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -639,6 +639,7 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test, if (ret) return ret; + uts->arg_error = false; ret = test->func(uts); if (ret == -EAGAIN) skip_test(uts); diff --git a/test/ut.c b/test/ut.c index aed59cae0b9..fe9a177ab53 100644 --- a/test/ut.c +++ b/test/ut.c @@ -295,3 +295,69 @@ void ut_set_skip_delays(struct unit_test_state *uts, bool skip_delays) state_set_skip_delays(skip_delays); #endif } + +const char *ut_get_str(struct unit_test_state *uts, int n, const char *file, + int line, const char *func) +{ + if (n < 0 || n >= uts->arg_count) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_str() arg check", + "arg %d is invalid (arg_count=%d)", n, + uts->arg_count); + uts->arg_error = true; + return NULL; + } + if (uts->args[n].type != UT_ARG_STR) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_str() type check", + "arg %d is not a string", n); + uts->arg_error = true; + return NULL; + } + + return uts->args[n].vstr; +} + +long ut_get_int(struct unit_test_state *uts, int n, const char *file, + int line, const char *func) +{ + if (n < 0 || n >= uts->arg_count) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_int() arg check", + "arg %d is invalid (arg_count=%d)", n, + uts->arg_count); + uts->arg_error = true; + return 0; + } + if (uts->args[n].type != UT_ARG_INT) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_int() type check", + "arg %d is not an int", n); + uts->arg_error = true; + return 0; + } + + return uts->args[n].vint; +} + +bool ut_get_bool(struct unit_test_state *uts, int n, const char *file, + int line, const char *func) +{ + if (n < 0 || n >= uts->arg_count) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_bool() arg check", + "arg %d is invalid (arg_count=%d)", n, + uts->arg_count); + uts->arg_error = true; + return false; + } + if (uts->args[n].type != UT_ARG_BOOL) { + if (!uts->arg_error) + ut_failf(uts, file, line, func, "ut_bool() type check", + "arg %d is not a bool", n); + uts->arg_error = true; + return false; + } + + return uts->args[n].vbool; +} -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add a priv[] buffer to struct unit_test_state that tests can use for their own data. This avoids the need to allocate memory or use global variables for test-specific state. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/tests_writing.rst | 14 ++++++++++++++ include/test/test.h | 3 +++ 2 files changed, 17 insertions(+) diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index 41612a9e21b..6842533e2c6 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -486,6 +486,20 @@ ut_check_delta(last) Return the change in free memory since ``last`` was obtained from ``ut_check_free()``. A positive value means more memory has been allocated. +Private buffer +~~~~~~~~~~~~~~ + +Each test has access to a private buffer ``uts->priv`` (256 bytes) for temporary +data. This avoids the need to allocate memory or use global variables:: + + static int my_test(struct unit_test_state *uts) + { + snprintf(uts->priv, sizeof(uts->priv), "/%s", filename); + /* use uts->priv as a path string */ + + return 0; + } + Writing Python tests -------------------- diff --git a/include/test/test.h b/include/test/test.h index 15ed8b37890..2facf5b3675 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -10,6 +10,7 @@ #include <linux/bitops.h> #define UT_MAX_ARGS 8 +#define UT_PRIV_SIZE 256 /** * struct ut_stats - Statistics about tests run @@ -95,6 +96,7 @@ struct ut_arg { * @args: Parsed argument values for current test * @arg_count: Number of parsed arguments * @arg_error: Set if ut_str/int/bool() detects a type mismatch + * @priv: Private data for tests to use as needed */ struct unit_test_state { struct ut_stats cur; @@ -124,6 +126,7 @@ struct unit_test_state { struct ut_arg args[UT_MAX_ARGS]; int arg_count; bool arg_error; + char priv[UT_PRIV_SIZE]; }; /* Test flags for each test */ -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Update the 'ut' command to have a -R command option to prevent ut_fail() and ut_failf() from clearing GD_FLG_RECORD. This is useful when testing the test framework itself, where error messages need to be captured. Refactor ut_fail() and ut_failf() to call ut_unsilence_console() instead of duplicating the flag-clearing logic. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/test/test.h | 2 ++ test/cmd_ut.c | 8 +++++++- test/ut.c | 7 ++++--- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/test/test.h b/include/test/test.h index 2facf5b3675..5ae90e39e00 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -96,6 +96,7 @@ struct ut_arg { * @args: Parsed argument values for current test * @arg_count: Number of parsed arguments * @arg_error: Set if ut_str/int/bool() detects a type mismatch + * @keep_record: Preserve console recording when ut_fail() is called * @priv: Private data for tests to use as needed */ struct unit_test_state { @@ -126,6 +127,7 @@ struct unit_test_state { struct ut_arg args[UT_MAX_ARGS]; int arg_count; bool arg_error; + bool keep_record; char priv[UT_PRIV_SIZE]; }; diff --git a/test/cmd_ut.c b/test/cmd_ut.c index adc96fcbcdc..6358f27a64f 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -252,6 +252,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) struct unit_test_state uts; bool show_suites = false; bool force_run = false; + bool keep_record = false; int runs_per_text = 1; struct suite *ste; char *name; @@ -276,6 +277,9 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) if (!strchr(test_insert, ':')) return CMD_RET_USAGE; break; + case 'R': + keep_record = true; + break; case 's': show_suites = true; break; @@ -288,6 +292,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) return CMD_RET_USAGE; ut_init_state(&uts); + uts.keep_record = keep_record; name = argv[0]; select_name = cmd_arg1(argc, argv); @@ -333,10 +338,11 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) } U_BOOT_LONGHELP(ut, - "[-rs] [-f] [-I<n>:<one_test>] <suite> [<test> [<args>...]] - run unit tests\n" + "[-rs] [-f] [-R] [-I<n>:<one_test>] <suite> [<test> [<args>...]] - run unit tests\n" " -r<runs> Number of times to run each test\n" " -f Force 'manual' tests to run as well\n" " -I Test to run after <n> other tests have run\n" + " -R Preserve console recording on test failure\n" " -s Show all suites with ut info\n" " <suite> Test suite to run (or comma-separated list)\n" " <test> Specific test to run (optional)\n" diff --git a/test/ut.c b/test/ut.c index fe9a177ab53..0677e3fbee9 100644 --- a/test/ut.c +++ b/test/ut.c @@ -23,7 +23,7 @@ DECLARE_GLOBAL_DATA_PTR; void ut_fail(struct unit_test_state *uts, const char *fname, int line, const char *func, const char *cond) { - gd->flags &= ~(GD_FLG_SILENT | GD_FLG_RECORD); + ut_unsilence_console(uts); printf("%s:%d, %s(): %s\n", fname, line, func, cond); uts->cur.fail_count++; } @@ -33,7 +33,7 @@ void ut_failf(struct unit_test_state *uts, const char *fname, int line, { va_list args; - gd->flags &= ~(GD_FLG_SILENT | GD_FLG_RECORD); + ut_unsilence_console(uts); printf("%s:%d, %s(): %s: ", fname, line, func, cond); va_start(args, fmt); vprintf(fmt, args); @@ -286,7 +286,8 @@ void ut_silence_console(struct unit_test_state *uts) void ut_unsilence_console(struct unit_test_state *uts) { - gd->flags &= ~(GD_FLG_SILENT | GD_FLG_RECORD); + if (!uts->keep_record) + gd->flags &= ~(GD_FLG_SILENT | GD_FLG_RECORD); } void ut_set_skip_delays(struct unit_test_state *uts, bool skip_delays) -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add a set of tests to check the behaviour of test arguments and the ut command. This includes failure cases, where the wrong type or a non-existent argument is requested. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- test/common/Makefile | 1 + test/common/test_args.c | 186 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 test/common/test_args.c diff --git a/test/common/Makefile b/test/common/Makefile index 9674bbec030..eba4f8c125f 100644 --- a/test/common/Makefile +++ b/test/common/Makefile @@ -16,3 +16,4 @@ obj-y += cread.o obj-y += malloc.o obj-$(CONFIG_CONSOLE_PAGER) += pager.o obj-$(CONFIG_$(PHASE_)CMDLINE) += print.o +obj-$(CONFIG_$(PHASE_)CMDLINE) += test_args.o diff --git a/test/common/test_args.c b/test/common/test_args.c new file mode 100644 index 00000000000..745caeb8991 --- /dev/null +++ b/test/common/test_args.c @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Test for unit test arguments + * + * Copyright 2025 Canonical Ltd + * Written by Simon Glass <simon.glass@canonical.com> + */ + +#include <string.h> +#include <test/common.h> +#include <test/test.h> +#include <test/ut.h> + +/* Test that string arguments work correctly */ +static int test_args_str_norun(struct unit_test_state *uts) +{ + ut_asserteq_str("hello", ut_str(0)); + + return 0; +} +UNIT_TEST_ARGS(test_args_str_norun, UTF_CONSOLE | UTF_MANUAL, common, + { "strval", UT_ARG_STR }); + +/* Test that integer arguments work correctly */ +static int test_args_int_norun(struct unit_test_state *uts) +{ + ut_asserteq(1234, ut_int(0)); + + return 0; +} +UNIT_TEST_ARGS(test_args_int_norun, UTF_CONSOLE | UTF_MANUAL, common, + { "intval", UT_ARG_INT }); + +/* Test that boolean arguments work correctly */ +static int test_args_bool_norun(struct unit_test_state *uts) +{ + ut_asserteq(true, ut_bool(0)); + + return 0; +} +UNIT_TEST_ARGS(test_args_bool_norun, UTF_CONSOLE | UTF_MANUAL, common, + { "boolval", UT_ARG_BOOL }); + +/* Test multiple arguments of different types */ +static int test_args_multi_norun(struct unit_test_state *uts) +{ + ut_asserteq_str("test", ut_str(0)); + ut_asserteq(42, ut_int(1)); + ut_asserteq(true, ut_bool(2)); + + return 0; +} +UNIT_TEST_ARGS(test_args_multi_norun, UTF_CONSOLE | UTF_MANUAL, common, + { "str", UT_ARG_STR }, + { "num", UT_ARG_INT }, + { "flag", UT_ARG_BOOL }); + +/* Test optional arguments with defaults */ +static int test_args_optional_norun(struct unit_test_state *uts) +{ + /* Required arg should match what was passed */ + ut_asserteq_str("required", ut_str(0)); + + /* Optional args should have default values if not provided */ + ut_asserteq(99, ut_int(1)); + ut_asserteq(false, ut_bool(2)); + + return 0; +} +UNIT_TEST_ARGS(test_args_optional_norun, UTF_CONSOLE | UTF_MANUAL, common, + { "req", UT_ARG_STR }, + { "opt_int", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 99 } }, + { "opt_bool", UT_ARG_BOOL, UT_ARGF_OPTIONAL, { .vbool = false } }); + +/* + * Test requesting wrong type - ut_int() on a string arg should fail + * This test deliberately causes a type mismatch to verify error handling + */ +static int test_args_wrongtype_norun(struct unit_test_state *uts) +{ + /* This should fail - asking for int but arg is string */ + ut_asserteq(0, ut_int(0)); + ut_asserteq(true, uts->arg_error); + + return 0; +} +UNIT_TEST_ARGS(test_args_wrongtype_norun, UTF_MANUAL, common, + { "strval", UT_ARG_STR }); + +/* + * Test requesting invalid arg number - ut_str(1) when only arg 0 exists + * This test deliberately causes an out-of-bounds access to verify error handling + */ +static int test_args_badnum_norun(struct unit_test_state *uts) +{ + /* This should fail - asking for arg 1 but only arg 0 exists */ + ut_asserteq_ptr(NULL, ut_str(1)); + ut_asserteq(true, uts->arg_error); + + return 0; +} +UNIT_TEST_ARGS(test_args_badnum_norun, UTF_MANUAL, common, + { "strval", UT_ARG_STR }); + +/* Wrapper test that runs the manual tests with proper arguments */ +static int test_args(struct unit_test_state *uts) +{ + ut_assertok(run_command("ut -f common test_args_str_norun strval=hello", + 0)); + ut_assertok(run_command("ut -f common test_args_int_norun intval=1234", + 0)); + ut_assertok(run_command("ut -f common test_args_bool_norun boolval=1", + 0)); + ut_assertok(run_command("ut -f common test_args_multi_norun str=test num=42 flag=1", + 0)); + ut_assertok(run_command("ut -f common test_args_optional_norun req=required", + 0)); + + return 0; +} +COMMON_TEST(test_args, UTF_CONSOLE); + +/* + * Test argument-parsing failure cases - these should all fail + * + * Note: Running 'ut' within a test is not normal practice since do_ut() + * creates a new test state. But it works here for testing the argument + * parsing itself. + */ +static int test_args_fail(struct unit_test_state *uts) +{ + /* Missing required argument - should fail */ + ut_asserteq(1, run_command("ut -f common test_args_str_norun", 0)); + ut_assert_nextline("Missing required argument 'strval' for test 'test_args_str_norun'"); + ut_assert_nextline_regex("Tests run: 1,.*failures: 1"); + ut_assert_console_end(); + + /* Unknown argument name - should fail */ + ut_asserteq(1, run_command("ut -f common test_args_str_norun badarg=x", + 0)); + ut_assert_nextline("Unknown argument 'badarg' for test 'test_args_str_norun'"); + ut_assert_nextline_regex("Tests run: 1,.*failures: 1"); + ut_assert_console_end(); + + /* Invalid format (no = sign) - should fail */ + ut_asserteq(1, run_command("ut -f common test_args_str_norun strval", + 0)); + ut_assert_nextline("Invalid argument 'strval' (expected key=value)"); + ut_assert_nextline_regex("Tests run: 1,.*failures: 1"); + ut_assert_console_end(); + + return 0; +} +COMMON_TEST(test_args_fail, UTF_CONSOLE); + +/* Test that requesting wrong type fails - ut_int() on string arg */ +static int test_args_wrongtype(struct unit_test_state *uts) +{ + ut_asserteq(1, + run_command("ut -R -f common test_args_wrongtype_norun strval=hello", + 0)); + ut_assert_nextline("Test: test_args_wrongtype_norun: test_args.c"); + ut_assert_nextline_regex("test/common/test_args.c:.*, test_args_wrongtype_norun\\(\\): ut_int\\(\\) type check: arg 0 is not an int"); + ut_assert_nextline("Test 'test_args_wrongtype_norun' failed 1 times"); + ut_assert_nextline_regex("Tests run: 1,.*failures: 1"); + ut_assert_console_end(); + + return 0; +} +COMMON_TEST(test_args_wrongtype, UTF_CONSOLE); + +/* Test that requesting invalid arg number fails */ +static int test_args_badnum(struct unit_test_state *uts) +{ + ut_asserteq(1, + run_command("ut -R -f common test_args_badnum_norun strval=hello", + 0)); + ut_assert_nextline("Test: test_args_badnum_norun: test_args.c"); + ut_assert_nextline_regex("test/common/test_args.c:.*, test_args_badnum_norun\\(\\): ut_str\\(\\) arg check: arg 1 is invalid \\(arg_count=1\\)"); + ut_assert_nextline("Test 'test_args_badnum_norun' failed 1 times"); + ut_assert_nextline_regex("Tests run: 1,.*failures: 1"); + ut_assert_console_end(); + + return 0; +} +COMMON_TEST(test_args_badnum, UTF_CONSOLE); -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add C implementations of filesystem tests that can be called via the 'ut fs' command. These tests use UTF_MANUAL flag since they require external setup, i.e. creation of filesystem images. This covers the existing TestFsBasic tests. The tests use typed arguments (fs_type, fs_image, md5 values) passed via the command line. Add a few helpers to make the code easier to read. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- include/test/fs.h | 39 ++++ test/Makefile | 1 + test/cmd_ut.c | 2 + test/fs/Makefile | 3 + test/fs/fs_basic.c | 492 +++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 537 insertions(+) create mode 100644 include/test/fs.h create mode 100644 test/fs/Makefile create mode 100644 test/fs/fs_basic.c diff --git a/include/test/fs.h b/include/test/fs.h new file mode 100644 index 00000000000..58fc105a94a --- /dev/null +++ b/include/test/fs.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2025 Google LLC + */ + +#ifndef __TEST_FS_H +#define __TEST_FS_H + +#include <test/test.h> +#include <test/ut.h> + +/** + * FS_TEST() - Define a new filesystem test + * + * @name: Name of test function + * @flags: Flags for the test (see enum ut_flags) + */ +#define FS_TEST(_name, _flags) UNIT_TEST(_name, UTF_DM | (_flags), fs) + +/** + * FS_TEST_ARGS() - Define a filesystem test with inline arguments + * + * Like FS_TEST() but for tests that take arguments. + * The test can access arguments via uts->args[]. + * The NULL terminator is added automatically. + * + * Example: + * FS_TEST_ARGS(my_test, UTF_MANUAL, + * { "fs_type", UT_ARG_STR }, + * { "fs_image", UT_ARG_STR }); + * + * @name: Name of test function + * @flags: Flags for the test (see enum ut_flags) + * @...: Argument definitions (struct ut_arg_def initializers) + */ +#define FS_TEST_ARGS(_name, _flags, ...) \ + UNIT_TEST_ARGS(_name, UTF_DM | (_flags), fs, __VA_ARGS__) + +#endif /* __TEST_FS_H */ diff --git a/test/Makefile b/test/Makefile index f7ab9a36b2a..1edd931f3e8 100644 --- a/test/Makefile +++ b/test/Makefile @@ -22,6 +22,7 @@ obj-y += boot/ obj-$(CONFIG_UNIT_TEST) += common/ obj-$(CONFIG_UT_ENV) += env/ obj-$(CONFIG_UT_FDT_OVERLAY) += fdt_overlay/ +obj-$(CONFIG_SANDBOX) += fs/ obj-y += log/ else obj-$(CONFIG_SPL_UT_LOAD) += image/ diff --git a/test/cmd_ut.c b/test/cmd_ut.c index 6358f27a64f..8d8a2d763a2 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -61,6 +61,7 @@ SUITE_DECL(exit); SUITE_DECL(fdt); SUITE_DECL(fdt_overlay); SUITE_DECL(font); +SUITE_DECL(fs); SUITE_DECL(hush); SUITE_DECL(lib); SUITE_DECL(loadm); @@ -89,6 +90,7 @@ static struct suite suites[] = { SUITE(fdt, "fdt command"), SUITE(fdt_overlay, "device tree overlays"), SUITE(font, "font command"), + SUITE(fs, "filesystem tests"), SUITE(hush, "hush behaviour"), SUITE(lib, "library functions"), SUITE(loadm, "loadm command parameters and loading memory blob"), diff --git a/test/fs/Makefile b/test/fs/Makefile new file mode 100644 index 00000000000..5899be8e667 --- /dev/null +++ b/test/fs/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0+ + +obj-y += fs_basic.o diff --git a/test/fs/fs_basic.c b/test/fs/fs_basic.c new file mode 100644 index 00000000000..d3b18275fac --- /dev/null +++ b/test/fs/fs_basic.c @@ -0,0 +1,492 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Basic filesystem tests - C implementation for Python wrapper + * + * These tests are marked UTF_MANUAL and are intended to be called from + * test_basic.py which sets up the filesystem image and expected values. + * + * Copyright 2025 Google LLC + */ + +#include <command.h> +#include <dm.h> +#include <env.h> +#include <fs.h> +#include <fs_legacy.h> +#include <hexdump.h> +#include <image.h> +#include <linux/sizes.h> +#include <mapmem.h> +#include <test/fs.h> +#include <test/test.h> +#include <test/ut.h> +#include <u-boot/md5.h> + +/* Test constants matching fstest_defs.py */ +#define ADDR 0x01000008 + +/* + * Common argument indices. Each test declares only the arguments it needs, + * so indices 2+ vary per test - see comments in each test. + */ +#define FS_ARG_TYPE 0 /* fs_type: ext4, fat, exfat, fs_generic */ +#define FS_ARG_IMAGE 1 /* fs_image: path to filesystem image */ + +/* Common arguments for all filesystem tests (indices 0 and 1) */ +#define COMMON_ARGS \ + { "fs_type", UT_ARG_STR }, \ + { "fs_image", UT_ARG_STR } + +/** + * get_fs_type(uts) - Get filesystem type enum from test argument + * + * Reads the fs_type argument and returns the appropriate FS_TYPE_* enum value. + * + * Return: filesystem type enum + */ +static int get_fs_type(struct unit_test_state *uts) +{ + const char *fs_type = ut_str(FS_ARG_TYPE); + + if (!fs_type) + return FS_TYPE_ANY; + + if (!strcmp(fs_type, "ext4")) + return FS_TYPE_EXT; + if (!strcmp(fs_type, "fat")) + return FS_TYPE_FAT; + if (!strcmp(fs_type, "exfat")) + return FS_TYPE_EXFAT; + + /* fs_generic uses FS_TYPE_ANY */ + return FS_TYPE_ANY; +} + +/* Set up the host filesystem block device */ +static int set_fs(struct unit_test_state *uts) +{ + return fs_set_blk_dev("host", "0:0", get_fs_type(uts)); +} + +/* Build a path by prepending "/" to the leaf filename, with optional suffix */ +static const char *getpath(struct unit_test_state *uts, const char *leaf, + const char *suffix) +{ + snprintf(uts->priv, sizeof(uts->priv), "/%s%s", leaf, suffix ?: ""); + + return uts->priv; +} + +/** + * prep_fs() - Prepare filesystem for testing + * + * Binds the fs_image argument as host device 0, sets up the block device, + * and optionally returns a zeroed buffer. + * + * @uts: Unit test state + * @len: Length of buffer to allocate and zero, or 0 for none + * @bufp: Returns pointer to zeroed buffer, or NULL if @len is 0 + * Return: 0 on success, negative on error + */ +static int prep_fs(struct unit_test_state *uts, uint len, void **bufp) +{ + const char *fs_image = ut_str(FS_ARG_IMAGE); + + ut_assertnonnull(fs_image); + ut_assertok(run_commandf("host bind 0 %s", fs_image)); + ut_assertok(set_fs(uts)); + + if (len) { + *bufp = map_sysmem(ADDR, len); + memset(*bufp, '\0', len); + } + + return 0; +} + +/** + * fs_write_supported(uts) - Check if write is supported for current fs type + * + * Reads the fs_type argument and checks if write support is enabled + * for that filesystem type. + * + * Return: true if write is supported, false otherwise + */ +static bool fs_write_supported(struct unit_test_state *uts) +{ + const char *fs_type = ut_str(FS_ARG_TYPE); + + if (!fs_type) + return false; + + if (!strcmp(fs_type, "ext4")) + return IS_ENABLED(CONFIG_EXT4_WRITE); + if (!strcmp(fs_type, "fat")) + return IS_ENABLED(CONFIG_CMD_FAT_WRITE); + + /* fs_generic and exfat use generic write which is always available */ + return true; +} + +/** + * verify_md5() - Calculate MD5 of buffer and verify against expected + * + * Uses arg 3 (md5val) as the expected MD5 hex string. + * + * @uts: Unit test state + * @buf: Buffer to calculate MD5 of + * @len: Length of buffer + * + * Return: 0 if MD5 matches, -EINVAL otherwise + */ +static int verify_md5(struct unit_test_state *uts, const void *buf, size_t len) +{ + u8 digest[MD5_SUM_LEN], expected[MD5_SUM_LEN]; + const char *expected_hex = ut_str(3); + + ut_assertok(hex2bin(expected, expected_hex, MD5_SUM_LEN)); + + md5_wd(buf, len, digest, CHUNKSZ_MD5); + ut_asserteq_mem(expected, digest, MD5_SUM_LEN); + + return 0; +} + +/** + * Test Case 1 - ls command, listing root directory and invalid directory + */ +static int fs_test_ls_norun(struct unit_test_state *uts) +{ + const char *small = ut_str(2); + const char *big = ut_str(3); + struct fs_dir_stream *dirs; + struct fs_dirent *dent; + int found_big = 0, found_small = 0, found_subdir = 0; + + ut_assertok(prep_fs(uts, 0, NULL)); + + /* Test listing root directory */ + dirs = fs_opendir("/"); + ut_assertnonnull(dirs); + + while ((dent = fs_readdir(dirs))) { + if (!strcmp(dent->name, big)) { + found_big = 1; + ut_asserteq(FS_DT_REG, dent->type); + } else if (!strcmp(dent->name, small)) { + found_small = 1; + ut_asserteq(FS_DT_REG, dent->type); + } else if (!strcmp(dent->name, "SUBDIR")) { + found_subdir = 1; + ut_asserteq(FS_DT_DIR, dent->type); + } + } + fs_closedir(dirs); + + ut_asserteq(1, found_big); + ut_asserteq(1, found_small); + ut_asserteq(1, found_subdir); + + /* Test invalid directory returns error */ + ut_assertok(set_fs(uts)); + dirs = fs_opendir("/invalid_d"); + ut_assertnull(dirs); + + /* Test file exists */ + ut_assertok(set_fs(uts)); + ut_asserteq(1, fs_exists(small)); + + /* Test non-existent file */ + ut_assertok(set_fs(uts)); + ut_asserteq(0, fs_exists("nonexistent.file")); + + return 0; +} +FS_TEST_ARGS(fs_test_ls_norun, UTF_SCAN_FDT | UTF_CONSOLE | UTF_MANUAL, + COMMON_ARGS, { "small", UT_ARG_STR }, { "big", UT_ARG_STR }); + +/** + * Test Case 2 - size command for small file (1MB) + */ +static int fs_test_size_small_norun(struct unit_test_state *uts) +{ + const char *small = ut_str(2); + loff_t size; + + ut_assertok(prep_fs(uts, 0, NULL)); + ut_assertok(fs_size(getpath(uts, small, NULL), &size)); + ut_asserteq(SZ_1M, size); + + /* Test size via path with '..' */ + ut_assertok(set_fs(uts)); + snprintf(uts->priv, sizeof(uts->priv), "/SUBDIR/../%s", small); + ut_assertok(fs_size(uts->priv, &size)); + ut_asserteq(SZ_1M, size); + + return 0; +} +FS_TEST_ARGS(fs_test_size_small_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "small", UT_ARG_STR }); + +/** + * Test Case 3 - size command for large file (2500 MiB) + */ +static int fs_test_size_big_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t size; + + ut_assertok(prep_fs(uts, 0, NULL)); + ut_assertok(fs_size(getpath(uts, big, NULL), &size)); + ut_asserteq_64((loff_t)SZ_1M * 2500, size); /* 2500 MiB = 0x9c400000 */ + + return 0; +} +FS_TEST_ARGS(fs_test_size_big_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }); + +/** + * Test Case 4 - load small file, verify MD5 + */ +static int fs_test_load_small_norun(struct unit_test_state *uts) +{ + const char *small = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, small, NULL), ADDR, 0, 0, + &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_small_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 5 - load first 1MB of big file + */ +static int fs_test_load_big_first_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0, SZ_1M, + &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_big_first_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 6 - load last 1MB of big file (offset 0x9c300000) + */ +static int fs_test_load_big_last_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0x9c300000ULL, + SZ_1M, &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_big_last_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 7 - load 1MB from last 1MB chunk of 2GB (offset 0x7ff00000) + */ +static int fs_test_load_big_2g_last_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0x7ff00000ULL, + SZ_1M, &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_big_2g_last_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 8 - load first 1MB in 2GB region (offset 0x80000000) + */ +static int fs_test_load_big_2g_first_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0x80000000ULL, + SZ_1M, &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_big_2g_first_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 9 - load 1MB crossing 2GB boundary (offset 0x7ff80000) + */ +static int fs_test_load_big_2g_cross_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0x7ff80000ULL, + SZ_1M, &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_load_big_2g_cross_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 10 - load beyond file end (2MB from offset where only 1MB remains) + */ +static int fs_test_load_beyond_norun(struct unit_test_state *uts) +{ + const char *big = ut_str(2); + loff_t actread; + void *buf; + + ut_assertok(prep_fs(uts, SZ_2M, &buf)); /* 2MB buffer */ + + /* Request 2MB starting at 1MB before EOF - should get 1MB */ + ut_assertok(fs_legacy_read(getpath(uts, big, NULL), ADDR, 0x9c300000ULL, + SZ_2M, &actread)); + ut_asserteq(SZ_1M, actread); /* Only 1MB available */ + + return 0; +} +FS_TEST_ARGS(fs_test_load_beyond_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "big", UT_ARG_STR }); + +/** + * Test Case 11 - write file + */ +static int fs_test_write_norun(struct unit_test_state *uts) +{ + const char *small = ut_str(2); + + loff_t actread, actwrite; + void *buf; + + if (!fs_write_supported(uts)) + return -EAGAIN; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + + /* Read small file */ + ut_assertok(fs_legacy_read(getpath(uts, small, NULL), ADDR, 0, 0, + &actread)); + ut_asserteq(SZ_1M, actread); + + /* Write it back with new name */ + ut_assertok(set_fs(uts)); + ut_assertok(fs_write(getpath(uts, small, ".w"), ADDR, 0, SZ_1M, + &actwrite)); + ut_asserteq(SZ_1M, actwrite); + + /* Read back and verify MD5 */ + ut_assertok(set_fs(uts)); + memset(buf, '\0', SZ_1M); + ut_assertok(fs_legacy_read(getpath(uts, small, ".w"), ADDR, 0, 0, + &actread)); + ut_asserteq(SZ_1M, actread); + + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_write_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR }); + +/** + * Test Case 12 - write to "." directory (should fail) + */ +static int fs_test_write_dot_norun(struct unit_test_state *uts) +{ + loff_t actwrite; + + if (!fs_write_supported(uts)) + return -EAGAIN; + + ut_assertok(prep_fs(uts, 0, NULL)); + + /* Writing to "." should fail */ + ut_assert(fs_write("/.", ADDR, 0, 0x10, &actwrite)); + + return 0; +} +FS_TEST_ARGS(fs_test_write_dot_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS); + +/** + * Test Case 13 - write via "./" path + */ +static int fs_test_write_dotpath_norun(struct unit_test_state *uts) +{ + const char *small = ut_str(2); + loff_t actread, actwrite; + void *buf; + + if (!fs_write_supported(uts)) + return -EAGAIN; + + ut_assertok(prep_fs(uts, SZ_1M, &buf)); + + /* Read small file */ + ut_assertok(fs_legacy_read(getpath(uts, small, NULL), ADDR, 0, 0, + &actread)); + ut_asserteq(SZ_1M, actread); + + /* Write via "./" path */ + ut_assertok(set_fs(uts)); + snprintf(uts->priv, sizeof(uts->priv), "/./%s2", small); + ut_assertok(fs_write(uts->priv, ADDR, 0, SZ_1M, &actwrite)); + ut_asserteq(SZ_1M, actwrite); + + /* Read back via "./" path and verify */ + ut_assertok(set_fs(uts)); + memset(buf, '\0', SZ_1M); + ut_assertok(fs_legacy_read(uts->priv, ADDR, 0, 0, &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + /* Also verify via normal path */ + ut_assertok(set_fs(uts)); + memset(buf, '\0', SZ_1M); + ut_assertok(fs_legacy_read(getpath(uts, small, "2"), ADDR, 0, 0, + &actread)); + ut_asserteq(SZ_1M, actread); + ut_assertok(verify_md5(uts, buf, SZ_1M)); + + return 0; +} +FS_TEST_ARGS(fs_test_write_dotpath_norun, UTF_SCAN_FDT | UTF_MANUAL, + COMMON_ARGS, { "small", UT_ARG_STR }, { "md5val", UT_ARG_STR }); -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Update test_basic.py to call the new C-based filesystem tests via the 'ut' command. This allows tests to run more directly, calling the actual FS layer, without having to go through the command interface and worrying about which filesystem command to use. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- test/py/tests/test_fs/conftest.py | 4 +- test/py/tests/test_fs/test_basic.py | 346 ++++++++-------------------- 2 files changed, 98 insertions(+), 252 deletions(-) diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py index 0205048e73a..5fc796097e3 100644 --- a/test/py/tests/test_fs/conftest.py +++ b/test/py/tests/test_fs/conftest.py @@ -198,8 +198,6 @@ def fs_obj_basic(request, u_boot_config): volume file name and a list of MD5 hashes. """ fs_type = request.param - fs_cmd_prefix = fstype_to_prefix(fs_type) - fs_cmd_write = 'save' if fs_type == 'fs_generic' or fs_type == 'exfat' else 'write' fs_img = '' fs_ubtype = fstype_to_ubname(fs_type) @@ -289,7 +287,7 @@ def fs_obj_basic(request, u_boot_config): pytest.skip('Setup failed for filesystem: ' + fs_type + '. {}'.format(err)) return else: - yield [fs_ubtype, fs_cmd_prefix, fs_cmd_write, fs_img, md5val] + yield [fs_ubtype, fs_img, md5val] finally: call('rm -rf %s' % scratch_dir, shell=True) call('rm -f %s' % fs_img, shell=True) diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py index 88b163ce305..7f805d04dd5 100644 --- a/test/py/tests/test_fs/test_basic.py +++ b/test/py/tests/test_fs/test_basic.py @@ -6,298 +6,146 @@ """ This test verifies basic read/write operation on file system. + +Tests are implemented in C (test/fs/fs_basic.c) and called from here. +Python handles filesystem image setup and environment variable configuration. """ import pytest -import re -from fstest_defs import * +from fstest_defs import SMALL_FILE, BIG_FILE from fstest_helpers import assert_fs_integrity -@pytest.mark.boardspec('sandbox') -@pytest.mark.slow -class TestFsBasic(object): - def test_fs1(self, ubman, fs_obj_basic): - """ - Test Case 1 - ls command, listing a root directory and invalid directory - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic - with ubman.log.section('Test Case 1a - ls'): - # Test Case 1 - ls - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sls host 0:0' % fs_cmd_prefix]) - assert(re.search('2621440000 *%s' % BIG_FILE, ''.join(output))) - assert(re.search('1048576 *%s' % SMALL_FILE, ''.join(output))) - with ubman.log.section('Test Case 1b - ls (invalid dir)'): - # In addition, test with a nonexistent directory to see if we crash. - output = ubman.run_command( - '%sls host 0:0 invalid_d' % fs_cmd_prefix) - assert('' == output) +def run_c_test(ubman, fs_type, fs_img, test_name, small=None, big=None, + md5val=None): + """Run a C unit test with proper setup. - with ubman.log.section('Test Case 1c - test -e'): - # Test Case 1 - test -e - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - 'test -e host 0:0 1MB.file && echo PASS']) - assert('PASS' in ''.join(output)) + Args: + ubman (ConsoleBase): U-Boot console manager + fs_type (str): Filesystem type (ext4, fat, fs_generic, exfat) + fs_img (str): Path to filesystem image + test_name (str): Name of C test function (without _norun suffix) + small (str): Filename of small test file (optional) + big (str): Filename of big test file (optional) + md5val (str): Expected MD5 value for verification (optional) - with ubman.log.section('Test Case 1d - test -e (invalid file)'): - # In addition, test with a nonexistent file to see if we crash. - output = ubman.run_command( - 'test -e host 0:0 2MB.file || echo PASS') - assert('PASS' in ''.join(output)) + Returns: + bool: True if test passed, False otherwise + """ + # Build the command with arguments + cmd = f'ut -f fs {test_name}_norun fs_type={fs_type} fs_image={fs_img}' + if small: + cmd += f' small={small}' + if big: + cmd += f' big={big}' + if md5val: + cmd += f' md5val={md5val}' - def test_fs2(self, ubman, fs_obj_basic): - """ - Test Case 2 - size command for a small file - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic - with ubman.log.section('Test Case 2a - size (small)'): - # 1MB is 0x0010 0000 - # Test Case 2a - size of small file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%ssize host 0:0 /%s' % (fs_cmd_prefix, SMALL_FILE), - 'printenv filesize', - 'setenv filesize']) - assert('filesize=100000' in ''.join(output)) + # Run the C test + ubman.run_command(cmd) + + # Check result + result = ubman.run_command('echo $?') + return result.strip() == '0' + + +@pytest.mark.boardspec('sandbox') +@pytest.mark.slow +class TestFsBasic: + """Test basic filesystem operations via C unit tests.""" + + def test_fs1(self, ubman, fs_obj_basic): + """Test Case 1 - ls command, listing root and invalid directories""" + fs_type, fs_img, _ = fs_obj_basic + with ubman.log.section('Test Case 1 - ls'): + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_ls', + small=SMALL_FILE, big=BIG_FILE) - with ubman.log.section('Test Case 2b - size (/../<file>)'): - # Test Case 2b - size of small file via a path using '..' - output = ubman.run_command_list([ - '%ssize host 0:0 /SUBDIR/../%s' % (fs_cmd_prefix, SMALL_FILE), - 'printenv filesize', - 'setenv filesize']) - assert('filesize=100000' in ''.join(output)) + def test_fs2(self, ubman, fs_obj_basic): + """Test Case 2 - size command for a small file""" + fs_type, fs_img, _ = fs_obj_basic + with ubman.log.section('Test Case 2 - size (small)'): + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_small', + small=SMALL_FILE) def test_fs3(self, ubman, fs_obj_basic): - """ - Test Case 3 - size command for a large file - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 3 - size command for a large file""" + fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 3 - size (large)'): - # 2.5GB (1024*1024*2500) is 0x9C40 0000 - # Test Case 3 - size of big file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%ssize host 0:0 /%s' % (fs_cmd_prefix, BIG_FILE), - 'printenv filesize', - 'setenv filesize']) - assert('filesize=9c400000' in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big', + big=BIG_FILE) def test_fs4(self, ubman, fs_obj_basic): - """ - Test Case 4 - load a small file, 1MB - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 4 - load a small file, 1MB""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 4 - load (small)'): - # Test Case 4a - Read full 1MB of small file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 4b - Read full 1MB of small file - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[0] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_small', + small=SMALL_FILE, md5val=md5val[0]) def test_fs5(self, ubman, fs_obj_basic): - """ - Test Case 5 - load, reading first 1MB of 3GB file - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 5 - load, reading first 1MB of 3GB file""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 5 - load (first 1MB)'): - # Test Case 5a - First 1MB of big file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s %x 0x0' % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 5b - First 1MB of big file - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[1] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_first', + big=BIG_FILE, md5val=md5val[1]) def test_fs6(self, ubman, fs_obj_basic): - """ - Test Case 6 - load, reading last 1MB of 3GB file - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 6 - load, reading last 1MB of 3GB file""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 6 - load (last 1MB)'): - # fails for ext as no offset support - # Test Case 6a - Last 1MB of big file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s %x 0x9c300000' - % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 6b - Last 1MB of big file - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[2] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_last', + big=BIG_FILE, md5val=md5val[2]) def test_fs7(self, ubman, fs_obj_basic): - """ - Test Case 7 - load, 1MB from the last 1MB in 2GB - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 7 - load, 1MB from the last 1MB in 2GB""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 7 - load (last 1MB in 2GB)'): - # fails for ext as no offset support - # Test Case 7a - One from the last 1MB chunk of 2GB - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s %x 0x7ff00000' - % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 7b - One from the last 1MB chunk of 2GB - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[3] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, + 'fs_test_load_big_2g_last', + big=BIG_FILE, md5val=md5val[3]) def test_fs8(self, ubman, fs_obj_basic): - """ - Test Case 8 - load, reading first 1MB in 2GB - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 8 - load, reading first 1MB in 2GB""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 8 - load (first 1MB in 2GB)'): - # fails for ext as no offset support - # Test Case 8a - One from the start 1MB chunk from 2GB - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s %x 0x80000000' - % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 8b - One from the start 1MB chunk from 2GB - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[4] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, + 'fs_test_load_big_2g_first', + big=BIG_FILE, md5val=md5val[4]) def test_fs9(self, ubman, fs_obj_basic): - """ - Test Case 9 - load, 1MB crossing 2GB boundary - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 9 - load, 1MB crossing 2GB boundary""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 9 - load (crossing 2GB boundary)'): - # fails for ext as no offset support - # Test Case 9a - One 1MB chunk crossing the 2GB boundary - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s %x 0x7ff80000' - % (fs_cmd_prefix, ADDR, BIG_FILE, LENGTH), - 'printenv filesize']) - assert('filesize=100000' in ''.join(output)) - - # Test Case 9b - One 1MB chunk crossing the 2GB boundary - output = ubman.run_command_list([ - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[5] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, + 'fs_test_load_big_2g_cross', + big=BIG_FILE, md5val=md5val[5]) def test_fs10(self, ubman, fs_obj_basic): - """ - Test Case 10 - load, reading beyond file end'): - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 10 - load, reading beyond file end""" + fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 10 - load (beyond file end)'): - # Generic failure case - # Test Case 10 - 2MB chunk from the last 1MB of big file - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s 0x00200000 0x9c300000' - % (fs_cmd_prefix, ADDR, BIG_FILE), - 'printenv filesize', - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert('filesize=100000' in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_beyond', + big=BIG_FILE) def test_fs11(self, ubman, fs_obj_basic): - """ - Test Case 11 - write' - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 11 - write""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 11 - write'): - # Read 1MB from small file - # Write it back to test the writes - # Test Case 11a - Check that the write succeeded - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE), - '%s%s host 0:0 %x /%s.w $filesize' - % (fs_cmd_prefix, fs_cmd_write, ADDR, SMALL_FILE)]) - assert('1048576 bytes written' in ''.join(output)) - - # Test Case 11b - Check md5 of written to is same - # as the one read from - output = ubman.run_command_list([ - '%sload host 0:0 %x /%s.w' % (fs_cmd_prefix, ADDR, SMALL_FILE), - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[0] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write', + small=SMALL_FILE, md5val=md5val[0]) assert_fs_integrity(fs_type, fs_img) def test_fs12(self, ubman, fs_obj_basic): - """ - Test Case 12 - write to "." directory - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 12 - write to "." directory""" + fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 12 - write (".")'): - # Next test case checks writing a file whose dirent - # is the first in the block, which is always true for "." - # The write should fail, but the lookup should work - # Test Case 12 - Check directory traversal - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%s%s host 0:0 %x /. 0x10' - % (fs_cmd_prefix, fs_cmd_write, ADDR)]) - assert('Unable to write' in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dot') assert_fs_integrity(fs_type, fs_img) def test_fs13(self, ubman, fs_obj_basic): - """ - Test Case 13 - write to a file with "/./<filename>" - """ - fs_type,fs_cmd_prefix,fs_cmd_write,fs_img,md5val = fs_obj_basic + """Test Case 13 - write to a file with '/./<filename>'""" + fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 13 - write ("./<file>")'): - # Read 1MB from small file - # Write it via "same directory", i.e. "." dirent - # Test Case 13a - Check directory traversal - output = ubman.run_command_list([ - 'host bind 0 %s' % fs_img, - '%sload host 0:0 %x /%s' % (fs_cmd_prefix, ADDR, SMALL_FILE), - '%s%s host 0:0 %x /./%s2 $filesize' - % (fs_cmd_prefix, fs_cmd_write, ADDR, SMALL_FILE)]) - assert('1048576 bytes written' in ''.join(output)) - - # Test Case 13b - Check md5 of written to is same - # as the one read from - output = ubman.run_command_list([ - 'mw.b %x 00 100' % ADDR, - '%sload host 0:0 %x /./%s2' % (fs_cmd_prefix, ADDR, SMALL_FILE), - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[0] in ''.join(output)) - - # Test Case 13c - Check md5 of written to is same - # as the one read from - output = ubman.run_command_list([ - 'mw.b %x 00 100' % ADDR, - '%sload host 0:0 %x /%s2' % (fs_cmd_prefix, ADDR, SMALL_FILE), - 'md5sum %x $filesize' % ADDR, - 'setenv filesize']) - assert(md5val[0] in ''.join(output)) + assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dotpath', + small=SMALL_FILE, md5val=md5val[0]) assert_fs_integrity(fs_type, fs_img) -- 2.43.0
From: Simon Glass <simon.glass@canonical.com> Add a description of how test parameters work. This helps to make it easier to write C tests which need setup to be done in Python. Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com> --- doc/develop/tests_writing.rst | 41 +++++++++++++++++++++++++++++++++++ doc/usage/cmd/ut.rst | 11 ++++++++-- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index 6842533e2c6..d322cf60e26 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the Python test. +Passing arguments to C tests +---------------------------- + +Sometimes a C test needs parameters from Python, such as filenames or expected +values that are generated at runtime. The test-argument feature allows this. + +Use the `UNIT_TEST_ARGS` macro to declare a test with arguments:: + + static int my_test_norun(struct unit_test_state *uts) + { + const char *filename = ut_str(0); + int count = ut_int(1); + + /* test code using filename and count */ + + return 0; + } + UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite, + { "filename", UT_ARG_STR }, + { "count", UT_ARG_INT }); + +Each argument definition specifies a name and type: + +- `UT_ARG_STR` - string argument, accessed via `ut_str(n)` +- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)` +- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)` + (use `1` for true, any other value for false) + +Arguments are passed on the command line in `name=value` format:: + + ut -f my_suite my_test_norun filename=/path/to/file count=42 + +From Python, you can call the test like this:: + + cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}' + ubman.run_command(cmd) + +This approach combines Python's flexibility for setup (creating files, +generating values) with C's speed and debuggability for the actual test logic. + + How slow are Python tests? -------------------------- diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index d8c3cbf496c..a26ee6ad7de 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...] + ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...] ut [-s] info Description @@ -37,10 +37,17 @@ test causes another test to fail. If the one test fails, testing stops immediately. +-R + Preserve console recording on test failure. Normally when a test fails, + console recording is disabled so error messages go directly to output. + This flag keeps recording enabled, which is useful when testing the test + framework itself. + args Optional arguments to pass to the test, in `name=value` format. These are used by tests declared with `UNIT_TEST_ARGS()` which define expected - argument names and types. + argument names and types. See :ref:`develop/tests_writing:passing arguments + to c tests` for details. Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it includes a near-complete set of emulators, no code-size limits, many CONFIG -- 2.43.0
participants (1)
-
Simon Glass