From 3ae9764b500e41b9fa04af1146a513f5c1708efd Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 10:21:01 -0400 Subject: [PATCH 1/7] feat(system,monitor): system info / reboot service and heap + task monitor service with a browser console New `system` component: - SystemInfo: static getters (and a collect() snapshot + to_string()) for the chip, ESP-IDF version, app description (project, version, build date/time, ELF SHA-256), running/boot partitions + OTA state, reset reason, uptime, MAC, flash/PSRAM size, CPU MHz and heap. - SystemControl: reboot(), and reboot_to_bootloader() which sets the chip's force-download-boot flag (RTC_CNTL on S2/S3/C3/C2, LP_AON on C6/H2/C5/ C61/H21, LP_SYSTEM on P4; classic ESP32 reports not supported) and restarts, so the device comes back in the ROM download mode; the S2/S3 ROM USB device is kept persistent across the reset. - SystemService: dispatcher module (`espp.system` v1, module 7): GET_INFO as tagged records hosts decode while skipping unknown tags, REBOOT and REBOOT_TO_BOOTLOADER (OK first, restart after a clamped delay) guarded by allow_reboot / allow_bootloader and an on_reboot_request veto callback. - Host-buildable codec (detail/system_protocol.hpp) with golden tests, docs, README, Doxygen wiring, and an esp32s3 example (vendor + CDC) in the CI matrix. `monitor` component: - MonitorService: dispatcher module (`espp.monitor` v1, module 8) serving HeapMonitor regions (GET_HEAP), the TaskMonitor table (GET_TASKS, capped at the frame payload) and a SET_STREAM periodic push of either; codec in detail/monitor_protocol.hpp with host tests. monitor now REQUIRES stream_frame + dispatcher. Web: components/system/web/system_console.html (WebUSB / Web Serial): device info, reboot / reboot-into-bootloader with an in-page confirmation (bootloader disabled when the device reports no capability), heap gauges and a live sortable task table with a stream toggle; resolves both module ids from discovery (monitor optional) with the shared helper block, added to the console lint. Module tables and web_apps docs updated. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- .github/workflows/build.yml | 3 + components/dispatcher/README.md | 2 + .../web/test/resolve_module_id_test.js | 1 + components/monitor/CMakeLists.txt | 4 +- components/monitor/README.md | 16 + components/monitor/idf_component.yml | 6 +- .../include/detail/monitor_protocol.hpp | 240 +++ .../monitor/include/monitor_service.hpp | 314 ++++ components/monitor/test/monitor_host_test.cpp | 151 ++ components/system/CMakeLists.txt | 18 + components/system/README.md | 72 + components/system/example/CMakeLists.txt | 46 + components/system/example/README.md | 58 + components/system/example/main/CMakeLists.txt | 5 + .../system/example/main/system_example.cpp | 115 ++ components/system/example/sdkconfig.defaults | 27 + components/system/idf_component.yml | 28 + .../system/include/detail/system_protocol.hpp | 350 +++++ components/system/include/system_control.hpp | 134 ++ components/system/include/system_info.hpp | 297 ++++ components/system/include/system_service.hpp | 284 ++++ components/system/test/system_host_test.cpp | 170 ++ components/system/web/system_console.html | 1361 +++++++++++++++++ doc/Doxyfile | 5 + doc/en/core/monitor.rst | 26 + doc/en/dispatcher/custom_modules.rst | 4 +- doc/en/dispatcher/dispatcher.rst | 2 + doc/en/index.rst | 1 + doc/en/system/index.rst | 16 + doc/en/system/system.rst | 61 + doc/en/system/system_example.md | 42 + doc/en/web_apps.rst | 5 + 32 files changed, 3861 insertions(+), 3 deletions(-) create mode 100644 components/monitor/include/detail/monitor_protocol.hpp create mode 100644 components/monitor/include/monitor_service.hpp create mode 100644 components/monitor/test/monitor_host_test.cpp create mode 100644 components/system/CMakeLists.txt create mode 100644 components/system/README.md create mode 100644 components/system/example/CMakeLists.txt create mode 100644 components/system/example/README.md create mode 100644 components/system/example/main/CMakeLists.txt create mode 100644 components/system/example/main/system_example.cpp create mode 100644 components/system/example/sdkconfig.defaults create mode 100644 components/system/idf_component.yml create mode 100644 components/system/include/detail/system_protocol.hpp create mode 100644 components/system/include/system_control.hpp create mode 100644 components/system/include/system_info.hpp create mode 100644 components/system/include/system_service.hpp create mode 100644 components/system/test/system_host_test.cpp create mode 100644 components/system/web/system_console.html create mode 100644 doc/en/system/index.rst create mode 100644 doc/en/system/system.rst create mode 100644 doc/en/system/system_example.md diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 4897c90b11..5d22020fd4 100755 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -130,6 +130,9 @@ jobs: - path: 'components/coredump/example' target: esp32s3 command: 'IDF_COMPONENT_MANAGER=0 idf.py build' + - path: 'components/system/example' + target: esp32s3 + command: 'IDF_COMPONENT_MANAGER=0 idf.py build' - path: 'components/cst816/example' target: esp32s3 - path: 'components/csv/example' diff --git a/components/dispatcher/README.md b/components/dispatcher/README.md index 19cc3709be..13231e90e9 100644 --- a/components/dispatcher/README.md +++ b/components/dispatcher/README.md @@ -29,6 +29,8 @@ protocols and examples use these ids by default: | 4 | crash dump | `espp.coredump` v1 | | 5 | CAN bridge | `espp.can-bridge` v1 | | 6 | MCP266 | `espp.mcp266` v1 | +| 7 | System info / reboot (`espp::SystemService`) | `espp.system` v1 | +| 8 | Heap / task monitor (`espp::MonitorService`) | `espp.monitor` v1 | | 0xF0–0xFE | reserved (meta) | — | | 0xFF | capability discovery | — | diff --git a/components/dispatcher/web/test/resolve_module_id_test.js b/components/dispatcher/web/test/resolve_module_id_test.js index 71c8b23849..b253a94683 100644 --- a/components/dispatcher/web/test/resolve_module_id_test.js +++ b/components/dispatcher/web/test/resolve_module_id_test.js @@ -24,6 +24,7 @@ const consoles = [ "components/mcp266/web/mcp266_console.html", "components/bldc_haptics/web/haptics_console.html", "components/telemetry/web/telemetry.html", + "components/system/web/system_console.html", ]; const hub = "components/dispatcher/web/dispatcher_hub.html"; diff --git a/components/monitor/CMakeLists.txt b/components/monitor/CMakeLists.txt index cb1e7ccab3..fb35c98f8a 100644 --- a/components/monitor/CMakeLists.txt +++ b/components/monitor/CMakeLists.txt @@ -1,4 +1,6 @@ +# stream_frame + dispatcher: the header-only MonitorService (monitor_service.hpp) +# that serves the heap / task statistics over a framed byte stream. idf_component_register( INCLUDE_DIRS "include" SRC_DIRS "src" - REQUIRES base_component task) + REQUIRES base_component task stream_frame dispatcher) diff --git a/components/monitor/README.md b/components/monitor/README.md index 30ed80fe2e..8c19212e0c 100644 --- a/components/monitor/README.md +++ b/components/monitor/README.md @@ -10,6 +10,7 @@ system. - [Monitor Component](#monitor-component) - [Task Monitor](#task-monitor) + - [Monitor Service](#monitor-service) - [Example](#example) @@ -24,6 +25,21 @@ There is an associated [task-monitor](https://github.com/esp-cpp/task-monitor) python gui which can parse the output of this component and render it as a chart or into a table for visualization. +## Monitor Service + +`espp::MonitorService` (`monitor_service.hpp`) serves the heap-region and task +statistics over any framed byte stream as an `espp::Dispatcher` module +(`espp.monitor` v1, module 8 by default): `GET_HEAP` (one record per configured +`MALLOC_CAP_*` region), `GET_TASKS` (the `TaskMonitor` table; needs +`CONFIG_FREERTOS_USE_TRACE_FACILITY` + `CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS`) +and `SET_STREAM` (periodic HEAP / TASKS events). The wire codec lives in +`include/detail/monitor_protocol.hpp` (host-buildable, tested by +`test/monitor_host_test.cpp`). The hosted +[system console](https://esp-cpp.github.io/espp/apps/system_console.html) web +app renders heap gauges and a live task table from it; the +[system](../system) component's example exposes it over USB together with +`espp::SystemService`. + ## Example This example shows how to use the `monitor` component to monitor the executing diff --git a/components/monitor/idf_component.yml b/components/monitor/idf_component.yml index 1e8bedd91e..ab886b1e0e 100644 --- a/components/monitor/idf_component.yml +++ b/components/monitor/idf_component.yml @@ -1,6 +1,6 @@ ## IDF Component Manager Manifest File license: "MIT" -description: "System Monitor component for ESP-IDF" +description: "System Monitor component for ESP-IDF: heap and task statistics, plus a framed stream service (MonitorService) that serves them over USB / WebUSB / Web Serial" url: "https://github.com/esp-cpp/espp/tree/main/components/monitor" repository: "git://github.com/esp-cpp/espp.git" maintainers: @@ -14,8 +14,12 @@ tags: - Monitor - Memory - Tasks + - WebUSB + - WebSerial dependencies: idf: version: '>=5.0' espp/base_component: '>=1.0' espp/task: '>=1.0' + espp/stream_frame: '>=1.0' + espp/dispatcher: '>=1.0' diff --git a/components/monitor/include/detail/monitor_protocol.hpp b/components/monitor/include/detail/monitor_protocol.hpp new file mode 100644 index 0000000000..24603efb31 --- /dev/null +++ b/components/monitor/include/detail/monitor_protocol.hpp @@ -0,0 +1,240 @@ +#pragma once + +// Wire protocol of espp::MonitorService: heap-region and task statistics over +// the espp stream_frame codec, routed by an espp::Dispatcher on module 8 by +// default (`espp.monitor` v1 through discovery). +// +// This header is deliberately host-buildable (stream_frame.hpp + the standard +// library only) so the codec is unit-tested on the host +// (components/monitor/test/monitor_host_test.cpp) and so host tools can reuse +// it. All multi-byte fields are little-endian; a `str` is [len u8][bytes]. +// +// Requests (host -> device, reply flag clear): +// 0x01 GET_HEAP (no payload) +// 0x02 GET_TASKS (no payload) +// 0x03 SET_STREAM [enable u8][period_ms u16][what u8: bit0 heap, bit1 tasks] +// enable = 0 stops the periodic HEAP / TASKS events; the +// device clamps the period to its Config::min_stream_period. +// Replies / events (device -> host, high bit set = frame reply flag): +// 0x81 HEAP [count u8]{[flags u32][free u32][min_free u32] +// [largest_free_block u32][allocated u32][total u32]} +// 0x82 TASKS [count u8]{[name str][cpu_percent u8][high_water_mark u32] +// [priority u8][core i8]} +// 0x83 OK [request_type u8] +// 0x84 ERROR [request_type u8][code u32][utf8 message] +// HEAP / TASKS answer the matching GET_* request and are also sent +// unsolicited while streaming is enabled (same encoding, so a host decodes +// both the same way). A TASKS payload is capped at the frame payload limit: +// tasks that would not fit are dropped from the END of the list. + +#include +#include +#include +#include +#include +#include + +#include "stream_frame.hpp" + +namespace espp::detail::monitor_protocol { + +/// Default dispatcher module id (a routing key only; see MonitorService::Config::module). +inline constexpr uint8_t kModule = 8; +/// Stable protocol identifier + version advertised through discovery. +inline constexpr const char *kProtocol = "espp.monitor"; +inline constexpr uint16_t kProtocolVersion = 1; + +/// Frame `type` values within the monitor module. +enum class Type : uint8_t { + // host -> device + GetHeap = 0x01, + GetTasks = 0x02, + SetStream = 0x03, + // device -> host (high bit set) + Heap = 0x81, + Tasks = 0x82, + Ok = 0x83, + Error = 0x84, +}; + +/// SET_STREAM `what` bits. +inline constexpr uint8_t kStreamHeap = 0x01; +inline constexpr uint8_t kStreamTasks = 0x02; + +/// One heap region as carried in a HEAP payload. +struct HeapRegion { + uint32_t flags{0}; ///< MALLOC_CAP_* bitmask the region was queried with. + uint32_t free_bytes{0}; + uint32_t min_free_bytes{0}; + uint32_t largest_free_block{0}; + uint32_t allocated_bytes{0}; + uint32_t total_size{0}; +}; + +/// One task as carried in a TASKS payload. +struct TaskEntry { + std::string name; + uint8_t cpu_percent{0}; + uint32_t high_water_mark{0}; + uint8_t priority{0}; + int8_t core_id{-2}; ///< 0 / 1, -1 = unpinned, -2 = unknown (core ids not compiled in). +}; + +/// Decoded SET_STREAM request. +struct StreamRequest { + bool enable{false}; + uint16_t period_ms{0}; + uint8_t what{0}; ///< kStreamHeap | kStreamTasks +}; + +/// Bytes one TaskEntry occupies on the wire (name capped at 255). +inline size_t task_entry_size(std::string_view name) { + return 1 + (name.size() > 255 ? 255 : name.size()) + 1 + 4 + 1 + 1; +} + +/// Append a [len u8][bytes] string (truncated to 255 bytes). +inline void put_str(std::vector &out, std::string_view s) { + const size_t n = s.size() > 255 ? 255 : s.size(); + out.push_back(static_cast(n)); + out.insert(out.end(), s.begin(), s.begin() + static_cast(n)); +} + +/// Whether a type value is a device->host reply / event. +inline constexpr bool is_reply(Type type) { return (static_cast(type) & 0x80) != 0; } + +/// Build an encoded frame for a monitor message (device->host types map to the +/// frame reply flag). +inline std::vector build_frame(Type type, std::span payload = {}, + uint8_t module = kModule) { + return espp::stream_frame::build_frame(is_reply(type), module, static_cast(type), + payload); +} + +// ---- encoders --------------------------------------------------------------- + +/// Encode a HEAP payload. At most 255 regions are encoded. +inline std::vector encode_heap(std::span regions) { + std::vector p; + const size_t n = regions.size() > 255 ? 255 : regions.size(); + p.reserve(1 + 24 * n); + p.push_back(static_cast(n)); + for (size_t i = 0; i < n; ++i) { + const auto &r = regions[i]; + espp::stream_frame::put_u32(p, r.flags); + espp::stream_frame::put_u32(p, r.free_bytes); + espp::stream_frame::put_u32(p, r.min_free_bytes); + espp::stream_frame::put_u32(p, r.largest_free_block); + espp::stream_frame::put_u32(p, r.allocated_bytes); + espp::stream_frame::put_u32(p, r.total_size); + } + return p; +} + +/// Encode a TASKS payload, keeping it within @p max_bytes (the frame payload +/// limit by default): tasks that would not fit are dropped from the end. +/// @param[out] encoded_count Set to the number of tasks encoded, if non-null. +inline std::vector encode_tasks(std::span tasks, + size_t max_bytes = espp::stream_frame::kMaxPayloadSize, + size_t *encoded_count = nullptr) { + std::vector p; + p.push_back(0); // count, patched below + size_t n = 0; + for (const auto &t : tasks) { + if (n == 255 || p.size() + task_entry_size(t.name) > max_bytes) + break; + put_str(p, t.name); + p.push_back(t.cpu_percent); + espp::stream_frame::put_u32(p, t.high_water_mark); + p.push_back(t.priority); + p.push_back(static_cast(t.core_id)); + ++n; + } + p[0] = static_cast(n); + if (encoded_count) + *encoded_count = n; + return p; +} + +/// Encode a SET_STREAM request payload. +inline std::vector encode_set_stream(bool enable, uint16_t period_ms, uint8_t what) { + std::vector p; + p.push_back(enable ? 1 : 0); + espp::stream_frame::put_u16(p, period_ms); + p.push_back(what); + return p; +} + +/// Encode an OK payload. +inline std::vector encode_ok(uint8_t request_type) { return {request_type}; } + +/// Encode an ERROR payload. +inline std::vector encode_error(uint8_t request_type, uint32_t code, + std::string_view message) { + std::vector p; + p.push_back(request_type); + espp::stream_frame::put_u32(p, code); + p.insert(p.end(), message.begin(), message.end()); + return p; +} + +// ---- decoders (nullopt on a malformed payload) --------------------------------- + +inline std::optional> decode_heap(std::span p) { + if (p.empty()) + return std::nullopt; + const size_t n = p[0]; + if (p.size() < 1 + 24 * n) + return std::nullopt; + std::vector out; + out.reserve(n); + size_t i = 1; + for (size_t k = 0; k < n; ++k, i += 24) { + HeapRegion r; + r.flags = espp::stream_frame::get_u32(p.subspan(i)); + r.free_bytes = espp::stream_frame::get_u32(p.subspan(i + 4)); + r.min_free_bytes = espp::stream_frame::get_u32(p.subspan(i + 8)); + r.largest_free_block = espp::stream_frame::get_u32(p.subspan(i + 12)); + r.allocated_bytes = espp::stream_frame::get_u32(p.subspan(i + 16)); + r.total_size = espp::stream_frame::get_u32(p.subspan(i + 20)); + out.push_back(r); + } + return out; +} + +inline std::optional> decode_tasks(std::span p) { + if (p.empty()) + return std::nullopt; + const size_t n = p[0]; + std::vector out; + out.reserve(n); + size_t i = 1; + for (size_t k = 0; k < n; ++k) { + if (i >= p.size()) + return std::nullopt; + const size_t len = p[i++]; + if (i + len + 7 > p.size()) + return std::nullopt; + TaskEntry t; + t.name.assign(reinterpret_cast(p.data() + i), len); + i += len; + t.cpu_percent = p[i++]; + t.high_water_mark = espp::stream_frame::get_u32(p.subspan(i)); + i += 4; + t.priority = p[i++]; + t.core_id = static_cast(p[i++]); + out.push_back(std::move(t)); + } + return out; +} + +inline std::optional decode_set_stream(std::span p) { + if (p.size() < 4) + return std::nullopt; + StreamRequest r; + r.enable = p[0] != 0; + r.period_ms = espp::stream_frame::get_u16(p.subspan(1)); + r.what = p[3]; + return r; +} + +} // namespace espp::detail::monitor_protocol diff --git a/components/monitor/include/monitor_service.hpp b/components/monitor/include/monitor_service.hpp new file mode 100644 index 0000000000..a1d12f8538 --- /dev/null +++ b/components/monitor/include/monitor_service.hpp @@ -0,0 +1,314 @@ +#pragma once + +// espp::MonitorService -- heap-region and task statistics (espp::HeapMonitor / +// espp::TaskMonitor) as a transport-agnostic dispatcher module +// (detail/monitor_protocol.hpp is the wire spec). Same contract as the other +// espp services: requests are handled under an internal mutex, the `send` +// callback runs with that mutex released, frames for other modules / +// reply-flagged frames are ignored so the service shares a stream. +// +// Wiring (one line per transport): +// +// espp::MonitorService monitor_service({.send = [&](auto f) { usb.write_vendor(f); }}); +// dispatcher.register_module(monitor_service); // module 8 + discovery metadata + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "esp_heap_caps.h" + +#include "dispatcher.hpp" +#include "stream_frame.hpp" + +#include "base_component.hpp" +#include "detail/monitor_protocol.hpp" +#include "heap_monitor.hpp" +#include "task.hpp" +#include "task_monitor.hpp" + +namespace espp { + +/** + * @brief Serves heap-region and task statistics over any framed byte stream + * (dispatcher module 8 by default; see Config::module). + * + * GET_HEAP answers with one record per configured heap region + * (Config::heap_regions, MALLOC_CAP_* masks; regions that do not exist on the + * chip, i.e. report a total size of 0, are left out). GET_TASKS answers with + * espp::TaskMonitor's per-task table (name, CPU %, stack high-water mark, + * priority, core); it needs CONFIG_FREERTOS_USE_TRACE_FACILITY and + * CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS -- without them the list is empty + * (logged once), never an error. A TASKS payload is capped at the frame + * payload limit: tasks that do not fit are dropped from the end (logged). + * SET_STREAM starts (or stops) a task that sends HEAP and / or TASKS events + * periodically (period clamped to Config::min_stream_period), so a host can + * plot them live without polling. + * + * **Threading**: an internal mutex covers the parser and request handling; + * every outbound frame (replies and streamed events alike) is serialized on + * a send mutex held across the `send` callback, so frames never interleave + * and `send` never runs concurrently with itself. `send` must therefore not + * re-enter this object. + * + * \section monitor_service_ex1 MonitorService Example + * \snippet system_example.cpp system_example + */ +class MonitorService : public BaseComponent { +public: + using Stream = espp::stream_frame::StreamParser; + using Type = espp::detail::monitor_protocol::Type; + + /// Default dispatcher module id (8). A routing key only: Config::module serves + /// on any id, and hosts find it through discovery (by kProtocol). + static constexpr uint8_t kModule = espp::detail::monitor_protocol::kModule; + /// Stable protocol identifier + version advertised through discovery. + static constexpr const char *kProtocol = espp::detail::monitor_protocol::kProtocol; + static constexpr uint16_t kProtocolVersion = espp::detail::monitor_protocol::kProtocolVersion; + + /// Transmits one encoded frame to the host. + using send_fn = std::function frame)>; + + /// Configuration for the MonitorService. + struct Config { + send_fn send{nullptr}; ///< Transmits an encoded frame (required). + /// Dispatcher module id this instance answers on (and stamps on every + /// frame it sends). A routing key only (0x00..0xEF). + uint8_t module{kModule}; + /// Heap regions reported, as MALLOC_CAP_* masks (HeapMonitor::get_info). + std::vector heap_regions{MALLOC_CAP_DEFAULT, MALLOC_CAP_INTERNAL, MALLOC_CAP_SPIRAM}; + /// Shortest streaming period a host may request (SET_STREAM is clamped to it). + std::chrono::milliseconds min_stream_period{100}; + /// The streaming task (started on the first SET_STREAM enable). + Task::BaseConfig task_config{.name = "monitor_stream", .stack_size_bytes = 6 * 1024}; + espp::Logger::Verbosity log_level{espp::Logger::Verbosity::WARN}; ///< Logger verbosity. + }; + + /// @brief Construct the service. + explicit MonitorService(const Config &config) + : BaseComponent("MonitorService", config.log_level) + , config_(config) + , period_(config.min_stream_period) {} + + ~MonitorService() { stop_stream(); } + + /// @brief The dispatcher module id this service answers on (Config::module). + uint8_t module_id() const { return config_.module; } + + /// @brief Discovery metadata for registering this service on a Dispatcher. + Dispatcher::ModuleInfo module_info() const { + return {.name = "Monitor", + .app = "system_console.html", + .description = "Heap regions and task statistics", + .protocol = kProtocol, + .protocol_version = kProtocolVersion}; + } + + /// @brief Whether periodic HEAP / TASKS events are being sent. + bool streaming() const { return streaming_.load(); } + + /// @brief Stop streaming (also done by the destructor and by SET_STREAM 0). + void stop_stream() { + std::unique_ptr task; + { + std::lock_guard lock(mutex_); + streaming_.store(false); + task = std::move(task_); + } + if (task) + task->stop(); + } + + /** + * @brief Dispatcher entry point: handle one routed frame. Frames for other + * modules and reply-flagged frames are ignored, so this can be + * registered directly: `dispatcher.register_module(service)`. + */ + void handle(const espp::stream_frame::Frame &frame) { + if (frame.module != module_id() || frame.is_reply()) + return; + handle_frame(frame.type, frame.payload); + } + + /// @brief Feed received transport bytes (standalone use, without a Dispatcher). + void feed(std::span data) { + std::vector frames; + { + std::lock_guard lock(mutex_); + frames = parser_.feed(data); + } + for (const auto &frame : frames) + handle(frame); + } + + /// @brief Discard any partially-buffered frame bytes (standalone feed() use). + void reset_parser() { + std::lock_guard lock(mutex_); + parser_.reset(); + } + + /** + * @brief Handle one already-parsed request frame. + * @return true if the type belongs to the monitor protocol (a reply was + * sent), false if it was ignored. + */ + bool handle_frame(uint8_t type, std::span payload) { + namespace proto = espp::detail::monitor_protocol; + switch (static_cast(type)) { + case Type::GetHeap: + send_frame(proto::build_frame(Type::Heap, build_heap(), module_id())); + return true; + case Type::GetTasks: + send_frame(proto::build_frame(Type::Tasks, build_tasks(), module_id())); + return true; + case Type::SetStream: { + const auto req = proto::decode_set_stream(payload); + if (!req) { + send_error(type, std::errc::invalid_argument, + "malformed SET_STREAM (expected u8 enable, u16 period_ms, u8 what)"); + return true; + } + if (req->enable && (req->what & (proto::kStreamHeap | proto::kStreamTasks)) == 0) { + send_error(type, std::errc::invalid_argument, "SET_STREAM: nothing selected to stream"); + return true; + } + if (req->enable) + start_stream(std::chrono::milliseconds(req->period_ms), req->what); + else + stop_stream(); + logger_.debug("SET_STREAM enable={} period_ms={} what=0x{:02x}", req->enable, + period_.load().count(), req->what); + send_frame(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id())); + return true; + } + default: + return false; // not a monitor request: ignore so the service can share a stream + } + } + +protected: + /// The HEAP payload for the configured regions (regions with no memory left out). + std::vector build_heap() const { + namespace proto = espp::detail::monitor_protocol; + std::vector regions; + regions.reserve(config_.heap_regions.size()); + for (const int flags : config_.heap_regions) { + const HeapMonitor::HeapInfo hi = HeapMonitor::get_info(flags); + if (hi.total_size == 0) + continue; // e.g. MALLOC_CAP_SPIRAM on a chip without PSRAM + regions.push_back({.flags = static_cast(hi.heap_flags), + .free_bytes = static_cast(hi.free_bytes), + .min_free_bytes = static_cast(hi.min_free_bytes), + .largest_free_block = static_cast(hi.largest_free_block), + .allocated_bytes = static_cast(hi.allocated_bytes), + .total_size = static_cast(hi.total_size)}); + } + return proto::encode_heap(regions); + } + + /// The TASKS payload (capped at the frame payload limit). + std::vector build_tasks() { + namespace proto = espp::detail::monitor_protocol; + const auto infos = TaskMonitor::get_latest_info_vector(); +#if !(CONFIG_FREERTOS_USE_TRACE_FACILITY && CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS) + logger_.warn_rate_limited("task statistics need CONFIG_FREERTOS_USE_TRACE_FACILITY and " + "CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS; reporting no tasks"); +#endif + std::vector tasks; + tasks.reserve(infos.size()); + // (without the FreeRTOS stats Kconfig `infos` is provably empty; the + // conversion is still the right code for the configured build) + // cppcheck-suppress knownEmptyContainer + std::transform( + infos.begin(), infos.end(), std::back_inserter(tasks), [](const TaskMonitor::TaskInfo &t) { + return proto::TaskEntry{ + .name = t.name, + .cpu_percent = static_cast(t.cpu_percent > 100 ? 100 : t.cpu_percent), + .high_water_mark = t.high_water_mark, + .priority = static_cast(t.priority > 255 ? 255 : t.priority), + .core_id = static_cast(t.core_id)}; + }); + size_t encoded = 0; + auto payload = proto::encode_tasks(tasks, espp::stream_frame::kMaxPayloadSize, &encoded); + // cppcheck-suppress unsignedLessThanZero + if (encoded < tasks.size()) + logger_.warn_rate_limited("TASKS payload full: reporting {} of {} tasks", encoded, + tasks.size()); + return payload; + } + + void start_stream(std::chrono::milliseconds period, uint8_t what) { + std::lock_guard lock(mutex_); + period_.store(std::max(period, config_.min_stream_period)); + what_.store(what); + streaming_.store(true); + if (task_) + return; // already running: the new period / selection apply on its next wake + task_ = std::make_unique( + Task::Config{.callback = [this](std::mutex &m, + std::condition_variable &cv) { return stream_step(m, cv); }, + .task_config = config_.task_config}); + task_->start(); + } + + /// One streaming period: send the selected reports, then wait (interruptibly). + bool stream_step(std::mutex &m, std::condition_variable &cv) { + namespace proto = espp::detail::monitor_protocol; + if (streaming_.load()) { + const uint8_t what = what_.load(); + if (what & proto::kStreamHeap) + send_frame(proto::build_frame(Type::Heap, build_heap(), module_id())); + if (what & proto::kStreamTasks) + send_frame(proto::build_frame(Type::Tasks, build_tasks(), module_id())); + } + std::unique_lock lock(m); + cv.wait_for(lock, period_.load()); + return false; // keep running until stopped + } + + /// Transmit a frame. Serialized on send_mutex_ (held across the callback) so + /// a streamed event and a reply never interleave. Never called with mutex_ held. + void send_frame(const std::vector &frame) { + if (frame.empty()) + return; + std::lock_guard send_lock(send_mutex_); + if (!config_.send) { + logger_.warn_rate_limited("no send function configured; dropping a {}-byte frame", + frame.size()); + return; + } + config_.send(frame); + } + + void send_error(uint8_t request_type, std::errc errc, std::string_view message) { + namespace proto = espp::detail::monitor_protocol; + logger_.warn("{} (type 0x{:02x})", message, request_type); + send_frame(proto::build_frame( + Type::Error, proto::encode_error(request_type, static_cast(errc), message), + module_id())); + } + +private: + Config config_; + mutable std::mutex mutex_; ///< guards the parser and task_ + mutable std::mutex send_mutex_; ///< serializes every outbound frame across `send` + Stream parser_; + std::unique_ptr task_; + std::atomic streaming_{false}; + std::atomic what_{0}; + std::atomic period_; +}; + +// Compile-time check that the service keeps satisfying the dispatcher's module +// contract (module_id() / module_info() / handle(frame)). +static_assert(DispatcherModuleConcept); + +} // namespace espp diff --git a/components/monitor/test/monitor_host_test.cpp b/components/monitor/test/monitor_host_test.cpp new file mode 100644 index 0000000000..3e760e100b --- /dev/null +++ b/components/monitor/test/monitor_host_test.cpp @@ -0,0 +1,151 @@ +// Host-buildable unit tests for the MonitorService wire codec +// (include/detail/monitor_protocol.hpp). Build & run with: +// c++ -std=c++20 -I../include -I../../stream_frame/include monitor_host_test.cpp -o test && +// ./test +// +// The codec needs no ESP-IDF headers; these are golden encode/decode tests so +// the browser console and any host tool can rely on the byte layout. + +#include +#include +#include +#include +#include + +#include "detail/monitor_protocol.hpp" + +namespace mp = espp::detail::monitor_protocol; +namespace sf = espp::stream_frame; + +static int g_failures = 0; +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::printf(" FAIL: %s (line %d)\n", #cond, __LINE__); \ + ++g_failures; \ + } \ + } while (0) + +static void test_heap_roundtrip() { + std::printf("test_heap_roundtrip\n"); + std::vector regions = { + {.flags = 0x1800, + .free_bytes = 100000, + .min_free_bytes = 90000, + .largest_free_block = 65536, + .allocated_bytes = 200000, + .total_size = 300000}, + {.flags = 0x400, + .free_bytes = 1, + .min_free_bytes = 2, + .largest_free_block = 3, + .allocated_bytes = 4, + .total_size = 5}, + }; + const auto p = mp::encode_heap(regions); + CHECK(p.size() == 1 + 2 * 24); + CHECK(p[0] == 2); + // golden: first region, flags 0x1800 little-endian, free 100000 = 0x000186A0 + CHECK(p[1] == 0x00 && p[2] == 0x18 && p[3] == 0x00 && p[4] == 0x00); + CHECK(p[5] == 0xA0 && p[6] == 0x86 && p[7] == 0x01 && p[8] == 0x00); + const auto d = mp::decode_heap(p); + CHECK(d && d->size() == 2); + if (d && d->size() == 2) { + CHECK((*d)[0].flags == 0x1800 && (*d)[0].free_bytes == 100000 && (*d)[0].total_size == 300000); + CHECK((*d)[1].largest_free_block == 3 && (*d)[1].allocated_bytes == 4); + } + // truncated: declared 2 regions, only one present + CHECK(!mp::decode_heap(std::span(p.data(), 1 + 24))); + CHECK(!mp::decode_heap({})); + // empty list + const auto e = mp::encode_heap({}); + CHECK(e.size() == 1 && e[0] == 0); + CHECK(mp::decode_heap(e) && mp::decode_heap(e)->empty()); +} + +static void test_tasks_roundtrip_and_cap() { + std::printf("test_tasks_roundtrip_and_cap\n"); + std::vector tasks = { + {.name = "main", .cpu_percent = 12, .high_water_mark = 3000, .priority = 1, .core_id = 0}, + {.name = "IDLE1", .cpu_percent = 88, .high_water_mark = 500, .priority = 0, .core_id = 1}, + {.name = "tiT", .cpu_percent = 0, .high_water_mark = 1234, .priority = 18, .core_id = -1}, + }; + size_t encoded = 0; + const auto p = mp::encode_tasks(tasks, sf::kMaxPayloadSize, &encoded); + CHECK(encoded == 3 && p[0] == 3); + // golden first record: [4]"main"[12][0xB8 0x0B 0 0][1][0] + const uint8_t golden[] = {3, 4, 'm', 'a', 'i', 'n', 12, 0xB8, 0x0B, 0, 0, 1, 0}; + CHECK(p.size() >= sizeof(golden) && std::equal(golden, golden + sizeof(golden), p.begin())); + const auto d = mp::decode_tasks(p); + CHECK(d && d->size() == 3); + if (d && d->size() == 3) { + CHECK((*d)[1].name == "IDLE1" && (*d)[1].cpu_percent == 88 && (*d)[1].core_id == 1); + CHECK((*d)[2].core_id == -1 && (*d)[2].priority == 18 && (*d)[2].high_water_mark == 1234); + } + // truncated payload is rejected + CHECK(!mp::decode_tasks(std::span(p.data(), p.size() - 1))); + // the cap drops whole entries from the end + size_t n2 = 0; + const auto capped = mp::encode_tasks(tasks, 1 + mp::task_entry_size("main") + 3, &n2); + CHECK(n2 == 1 && capped[0] == 1 && capped.size() == 1 + mp::task_entry_size("main")); + // a 4096-byte payload never overflows: 500 tasks with long names + std::vector many(500, {.name = std::string(40, 'x'), + .cpu_percent = 1, + .high_water_mark = 1, + .priority = 1, + .core_id = 0}); + size_t n3 = 0; + const auto big = mp::encode_tasks(many, sf::kMaxPayloadSize, &n3); + CHECK(big.size() <= sf::kMaxPayloadSize && n3 < 500 && n3 == big[0]); + CHECK(mp::decode_tasks(big) && mp::decode_tasks(big)->size() == n3); + // a name longer than 255 bytes is truncated on the wire + std::vector longname = {{.name = std::string(300, 'n'), + .cpu_percent = 0, + .high_water_mark = 0, + .priority = 0, + .core_id = 0}}; + const auto ln = mp::encode_tasks(longname); + CHECK(mp::decode_tasks(ln) && (*mp::decode_tasks(ln))[0].name.size() == 255); +} + +static void test_set_stream_and_replies() { + std::printf("test_set_stream_and_replies\n"); + const auto p = mp::encode_set_stream(true, 250, mp::kStreamHeap | mp::kStreamTasks); + const uint8_t golden[] = {1, 0xFA, 0x00, 0x03}; + CHECK(p.size() == 4 && std::equal(golden, golden + 4, p.begin())); + const auto r = mp::decode_set_stream(p); + CHECK(r && r->enable && r->period_ms == 250 && r->what == 3); + CHECK(!mp::decode_set_stream(std::span(p.data(), 3))); + CHECK(mp::encode_ok(0x03) == std::vector{0x03}); + const auto e = mp::encode_error(0x02, 95, "no"); + const uint8_t eg[] = {0x02, 95, 0, 0, 0, 'n', 'o'}; + CHECK(e.size() == 7 && std::equal(eg, eg + 7, e.begin())); +} + +static void test_frames() { + std::printf("test_frames\n"); + // device->host types set the frame reply flag; requests do not + const auto req = mp::build_frame(mp::Type::GetHeap, {}, 8); + CHECK(req.size() == 9 + 4 && req[2] == 0x10 && req[3] == 8 && req[4] == 0x01); + const auto rep = mp::build_frame(mp::Type::Heap, mp::encode_heap({}), 9); + CHECK(rep[2] == 0x11 && rep[3] == 9 && rep[4] == 0x81); + sf::StreamParser parser; + std::vector stream(req); + stream.insert(stream.end(), rep.begin(), rep.end()); + const auto frames = parser.feed(stream); + CHECK(frames.size() == 2 && !frames[0].is_reply() && frames[1].is_reply() && + frames[1].module == 9 && frames[1].payload.size() == 1); +} + +int main() { + test_heap_roundtrip(); + test_tasks_roundtrip_and_cap(); + test_set_stream_and_replies(); + test_frames(); + if (g_failures) { + std::printf("%d FAILURE(S)\n", g_failures); + return 1; + } + std::printf("ALL TESTS PASSED\n"); + return 0; +} diff --git a/components/system/CMakeLists.txt b/components/system/CMakeLists.txt new file mode 100644 index 0000000000..d387f08077 --- /dev/null +++ b/components/system/CMakeLists.txt @@ -0,0 +1,18 @@ +# Header-only component (system_info.hpp, system_control.hpp, system_service.hpp). +# +# REQUIRES are public since the headers include theirs: +# esp_system — esp_system.h (esp_restart, esp_reset_reason, heap figures) +# esp_app_format — esp_app_desc.h (the embedded application description) +# app_update — esp_ota_ops.h (running / boot partition, OTA state) +# esp_hw_support — esp_chip_info.h, esp_mac.h, esp_clk_tree.h +# esp_timer — esp_timer.h (uptime) +# spi_flash — esp_flash.h (flash size) +# esp_psram — esp_psram.h (PSRAM size; empty component without PSRAM) +# soc / esp_rom — the always-on register + ROM USB persist calls behind +# SystemControl::reboot_to_bootloader() +# format — format.hpp (SystemInfo::to_string) +# stream_frame, dispatcher — the framed service (header-only, ESP-free codec) +idf_component_register( + INCLUDE_DIRS "include" + REQUIRES base_component esp_system esp_app_format app_update esp_hw_support esp_timer spi_flash esp_psram soc esp_rom format stream_frame dispatcher +) diff --git a/components/system/README.md b/components/system/README.md new file mode 100644 index 0000000000..5c02f16323 --- /dev/null +++ b/components/system/README.md @@ -0,0 +1,72 @@ +# System Component + +[![Badge](https://components.espressif.com/components/espp/system/badge.svg)](https://components.espressif.com/components/espp/system) + +The `system` component answers "what is this device, how is it doing, and can +I restart it?" for any espp application: + +- `espp::SystemInfo` — static getters (and a one-call `collect()` / + `to_string()` snapshot) for the chip model / revision / cores / features, the + ESP-IDF version, the application description (project name, version, build + date and time, ELF SHA-256), the running and boot partitions with the OTA + image state, the reset reason, uptime, base MAC, flash and PSRAM sizes, CPU + frequency and the free / minimum-free heap. +- `espp::SystemControl` — `reboot()` and `reboot_to_bootloader()`: the latter + sets the chip's *force download boot* flag and restarts, so the device comes + back in the ROM bootloader's download mode ready for `esptool` / `idf.py + flash` (what holding the BOOT strap does, without a button). Supported on the + ESP32-S2 / -S3 (the ROM's USB CDC / DFU device stays attached), -C2 / -C3 / + -C5 / -C6 / -C61 / -H2 / -H21 and -P4 (USB-Serial-JTAG); the classic ESP32 + has no software path and reports `operation_not_supported`. +- `espp::SystemService` — both of the above as a transport-agnostic + [dispatcher](../dispatcher) module (`espp.system` v1, module 7 by default): + `GET_INFO` answers with a tagged-record snapshot hosts can extend-proof + decode, `REBOOT` / `REBOOT_TO_BOOTLOADER` reply OK and restart after a delay. + Both restarts are guarded by `Config::allow_reboot` / `allow_bootloader` and + an optional `on_reboot_request` veto callback, so an application can refuse a + restart while, say, a motor is running. + +The hosted [system console](https://esp-cpp.github.io/espp/apps/system_console.html) +web app (`web/system_console.html`) talks to the service over WebUSB or Web +Serial, and to the [monitor](../monitor) component's `MonitorService` (heap +regions + task table, live) when the device advertises it. + + +**Table of Contents** + +- [System Component](#system-component) + - [Protocol (module 7, `espp.system` v1)](#protocol-module-7-esppsystem-v1) + - [Example](#example) + + + +## Protocol (module 7, `espp.system` v1) + +Framed with `stream_frame` and routed by `espp::Dispatcher`; requests carry the +reply flag clear, replies set it (type high bit). All multi-byte fields are +little-endian. See `include/detail/system_protocol.hpp` (host-buildable, with +`test/system_host_test.cpp`). + +| Type | Dir | Meaning | +|------|-----|---------| +| `0x01` GET_INFO | H→D | request an INFO reply | +| `0x02` REBOOT | H→D | `[delay_ms u16]` — reply OK, restart after the delay | +| `0x03` REBOOT_TO_BOOTLOADER | H→D | `[delay_ms u16]` — reply OK, restart into download mode | +| `0x81` INFO | D→H | tagged records `[tag u8][len u8][value]` (unknown tags are skipped) | +| `0x83` OK | D→H | `[request_type u8]` | +| `0x84` ERROR | D→H | `[request_type u8][code u32][utf8 message]` | + +INFO tags: 1 chip model (str), 2 chip revision (u16), 3 cores (u8), 4 chip +features (u32), 5 IDF version, 6 project name, 7 app version, 8 build date, 9 +build time, 10 ELF SHA-256 (32 bytes), 11 running partition, 12 boot partition, +13 OTA state (u8), 14 reset reason (u8), 15 uptime ms (u64), 16 MAC (6 bytes), +17 flash size, 18 PSRAM size, 19 CPU MHz, 20 free heap, 21 min free heap (all +u32), 22 capabilities (u32: bit0 reboot allowed, bit1 bootloader reboot allowed +and supported). The delay is clamped to at least `Config::min_restart_delay` +so the OK reply leaves the transport before the restart. + +## Example + +The [example](./example) exposes `SystemService` and `MonitorService` on the +native USB port (vendor / WebUSB + CDC / Web Serial) of an ESP32-S3 for the +system console web app. diff --git a/components/system/example/CMakeLists.txt b/components/system/example/CMakeLists.txt new file mode 100644 index 0000000000..28de1dafed --- /dev/null +++ b/components/system/example/CMakeLists.txt @@ -0,0 +1,46 @@ +# The following lines of boilerplate have to be in your project's CMakeLists +# in this exact order for cmake to work correctly +cmake_minimum_required(VERSION 3.20) + +set(CMAKE_CXX_STANDARD 23) + +# This example needs the managed `espressif/esp_tinyusb` component (required by +# usb_device). It supports two build modes: +# +# * DEFAULT (component manager ON) - how an end user builds it from the +# component registry: the manager fetches esp_tinyusb (and its tinyusb +# dependency) and the espp/* dependencies from the registry. +# EXTRA_COMPONENT_DIRS is narrowed to just the components this example uses. +# +# * MANAGER OFF (IDF_COMPONENT_MANAGER=0) - used by CI: every espp dependency +# resolves locally from EXTRA_COMPONENT_DIRS, and esp_tinyusb / tinyusb come +# from the vendored git submodules under external/. +include($ENV{IDF_PATH}/tools/cmake/project.cmake) + +set(EXTRA_COMPONENT_DIRS + "../../../components/base_component" + "../../../components/dispatcher" + "../../../components/format" + "../../../components/logger" + "../../../components/monitor" + "../../../components/stream_frame" + "../../../components/system" + "../../../components/task" + "../../../components/usb_device" +) + +if(DEFINED ENV{IDF_COMPONENT_MANAGER} AND "$ENV{IDF_COMPONENT_MANAGER}" STREQUAL "0") + list(APPEND EXTRA_COMPONENT_DIRS + "../../../external/esp-usb/device/esp_tinyusb" + "../../../external/tinyusb" + ) +endif() + +set( + COMPONENTS + "main esptool_py base_component dispatcher format logger monitor stream_frame system task usb_device esp_tinyusb esp_psram" + CACHE STRING + "List of components to include" + ) + +project(system_example) diff --git a/components/system/example/README.md b/components/system/example/README.md new file mode 100644 index 0000000000..e06afc0361 --- /dev/null +++ b/components/system/example/README.md @@ -0,0 +1,58 @@ +# System Info + Control over USB Example + +Exposes the espp `SystemService` (device identity / status, reboot, reboot +into the ROM bootloader) and `MonitorService` (heap regions and the task +table, on request or streamed) on the native USB port of an ESP32-S3, over +both the **vendor (WebUSB)** and **CDC (Web Serial)** interfaces. The hosted +[system console web app](https://esp-cpp.github.io/espp/apps/system_console.html) +(`components/system/web/system_console.html`) talks to both services; the +[Device Hub](https://esp-cpp.github.io/espp/apps/dispatcher_hub.html) lists them +through discovery (`espp.system` v1 on module 7, `espp.monitor` v1 on module 8 +by default). + +## How to use example + +### Hardware Required + +An ESP32-S3 (or -S2 / -P4) board with the native USB port wired to a host. The +system console / logs go to UART0 (see `sdkconfig.defaults`). + +### Build and Flash + +``` +idf.py set-target esp32s3 +idf.py build flash monitor +``` + +CI builds it with the component manager off (`IDF_COMPONENT_MANAGER=0 idf.py +build`), resolving every dependency from the repository (including the +vendored `esp_tinyusb` / `tinyusb` submodules under `external/`). + +Then open the system console web app and Connect (WebUSB or Web Serial): + +- **Device info**: chip, ESP-IDF version, application (project, version, build + date / time, ELF SHA-256), running / boot partition and OTA state, reset + reason, uptime, MAC, flash / PSRAM size, CPU frequency, heap. +- **Reboot** and **Reboot into bootloader**: the device replies OK and restarts + after the delay; in the second case it comes back in the ROM download mode + (the S3's ROM USB CDC / DFU interface) ready for `esptool` / `idf.py flash`. + The example's `on_reboot_request` callback logs and permits every request. +- **Heap** gauges per region and a **live task table** (CPU %, stack + high-water mark, priority, core) with a stream toggle and period. + +Task statistics need `CONFIG_FREERTOS_USE_TRACE_FACILITY` and +`CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS` (set in `sdkconfig.defaults`). + +## Example Output + +``` +I (317) System Example: Starting system info + control example +I (327) System Example: System: +ESP32-S3 rev 0.2 (2 cores), ESP-IDF v6.1 +app: system_example 1 built Sep 30 2026 12:34:56 +partition: running 'factory', boot 'factory', OTA state n/a +reset: power-on; uptime 320 ms; MAC 34:85:18:xx:xx:xx +flash 8192 KiB, PSRAM 0 KiB, CPU 240 MHz, heap free 318412 (min 318412) +I (357) System Example: Reboot into the bootloader is supported on this chip +I (1077) System Example: Ready. Connect the native USB port and open the system console ... +``` diff --git a/components/system/example/main/CMakeLists.txt b/components/system/example/main/CMakeLists.txt new file mode 100644 index 0000000000..57bd1ae9a2 --- /dev/null +++ b/components/system/example/main/CMakeLists.txt @@ -0,0 +1,5 @@ +idf_component_register( + SRC_DIRS "." + INCLUDE_DIRS "." + REQUIRES system monitor dispatcher task usb_device esp_tinyusb +) diff --git a/components/system/example/main/system_example.cpp b/components/system/example/main/system_example.cpp new file mode 100644 index 0000000000..a2340d9030 --- /dev/null +++ b/components/system/example/main/system_example.cpp @@ -0,0 +1,115 @@ +#include +#include +#include + +#include "sdkconfig.h" + +#include "dispatcher_worker.hpp" +#include "logger.hpp" +#include "monitor_service.hpp" +#include "system_control.hpp" +#include "system_info.hpp" +#include "system_service.hpp" +#include "usb_device.hpp" + +using namespace std::chrono_literals; + +// System info + control over USB example. +// +// Exposes two espp services on the USB vendor (WebUSB) and CDC (Web Serial) +// interfaces, routed by one espp::DispatcherWorker per transport: +// - espp::SystemService (module 7, `espp.system`): device identity / +// status, reboot, reboot into the ROM bootloader (download mode) +// - espp::MonitorService (module 8, `espp.monitor`): heap regions and the +// task table, on request or streamed +// The hosted system console web app (components/system/web/system_console.html) +// talks to both; the Device Hub lists them through discovery. + +extern "C" void app_main(void) { + espp::Logger logger({.tag = "System Example", .level = espp::Logger::Verbosity::INFO}); + logger.info("Starting system info + control example"); + + //! [system_example] + // Boot banner: everything SystemInfo knows, in one string. + logger.info("System:\n{}", espp::SystemInfo::to_string()); + logger.info("Reboot into the bootloader is {} on this chip", + espp::SystemControl::bootloader_reboot_supported() ? "supported" : "NOT supported"); + + // USB composite device: a vendor/WebUSB function and a CDC function, both + // carrying the same framed protocol. + espp::UsbDevice::Config usb_cfg; + usb_cfg.pid = 0x0d37; // distinct from the espp default so the webapp filter is specific + usb_cfg.manufacturer = "espp"; + usb_cfg.product = "espp System"; + usb_cfg.log_level = espp::Logger::Verbosity::WARN; + espp::UsbDevice::CdcFunction cdc; + cdc.interface_name = "espp System (CDC)"; + usb_cfg.cdc = cdc; + espp::UsbDevice::VendorFunction vendor; + vendor.interface_name = "espp System (WebUSB)"; + vendor.webusb = true; // advertise BOS / WebUSB / MS OS 2.0 descriptors + vendor.landing_page_url = "esp-cpp.github.io/espp/apps/system_console.html"; + usb_cfg.vendor = vendor; + espp::UsbDevice usb(usb_cfg); + + // Replies go back on the stream the request came in on: one send function + // per transport. Both services on one transport share it, and each service + // serializes its own frames; the USB writes are all-or-nothing per call. + auto vendor_send = [&](std::span frame) { usb.write_vendor(frame); }; + auto cdc_send = [&](std::span frame) { usb.write_cdc(frame); }; + + // The application decides whether a reboot may happen right now: this demo + // permits every request and logs it. A real application would refuse (or + // defer) while, say, a motor is running or a file is being written. + auto reboot_request = [&](espp::SystemService::RebootKind kind) { + logger.warn("Host requested a {}; allowing it", + kind == espp::SystemService::RebootKind::Bootloader ? "reboot into the bootloader" + : "reboot"); + return true; + }; + + // One service instance per transport (they are cheap; each replies on its + // own stream). Declared BEFORE the workers that call into them. + espp::SystemService vendor_system({.send = vendor_send, + .on_reboot_request = reboot_request, + .log_level = espp::Logger::Verbosity::INFO}); + espp::SystemService cdc_system({.send = cdc_send, + .on_reboot_request = reboot_request, + .log_level = espp::Logger::Verbosity::INFO}); + espp::MonitorService vendor_monitor( + {.send = vendor_send, .log_level = espp::Logger::Verbosity::INFO}); + espp::MonitorService cdc_monitor({.send = cdc_send, .log_level = espp::Logger::Verbosity::INFO}); + + // One DispatcherWorker per byte stream: a bounded receive queue + worker + // task feeding its Dispatcher, so the services never run on the TinyUSB + // task. Registering a service routes its module to it and advertises it for + // discovery; serve_discovery() answers the hub's ListModules query. + espp::DispatcherWorker vendor_link( + {.send = vendor_send, .task_config = {.name = "system_rx_vendor", .stack_size_bytes = 8192}}); + espp::DispatcherWorker cdc_link( + {.send = cdc_send, .task_config = {.name = "system_rx_cdc", .stack_size_bytes = 8192}}); + vendor_link.register_module(vendor_system); + vendor_link.register_module(vendor_monitor); + cdc_link.register_module(cdc_system); + cdc_link.register_module(cdc_monitor); + vendor_link.serve_discovery(usb_cfg.product); + cdc_link.serve_discovery(usb_cfg.product); + //! [system_example] + + // RX plumbing: the TinyUSB callbacks just queue the bytes for the workers. + usb.set_vendor_receive_callback([&](std::span data) { vendor_link.push(data); }); + usb.set_cdc_receive_callback([&](std::span data) { cdc_link.push(data); }); + + std::error_code usb_ec; + if (!usb.initialize(usb_ec)) + logger.error("Failed to initialize USB device: {}", usb_ec.message()); + else + logger.info("Ready. Connect the native USB port and open the system console " + "(components/system/web/system_console.html or https://{})", + vendor.landing_page_url); + + // Idle; all work happens in the dispatcher workers and the monitor stream task. + while (true) { + std::this_thread::sleep_for(1s); + } +} diff --git a/components/system/example/sdkconfig.defaults b/components/system/example/sdkconfig.defaults new file mode 100644 index 0000000000..bd0e1388d8 --- /dev/null +++ b/components/system/example/sdkconfig.defaults @@ -0,0 +1,27 @@ +CONFIG_IDF_TARGET="esp32s3" + +CONFIG_ESP_SYSTEM_EVENT_TASK_STACK_SIZE=4096 +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 + +# Console on UART0 (secondary on USB-Serial-JTAG): on the S3, USB-Serial-JTAG +# shares the native USB PHY with USB-OTG, which TinyUSB takes over. +CONFIG_ESP_CONSOLE_UART_DEFAULT=y +CONFIG_ESP_CONSOLE_SECONDARY_USB_SERIAL_JTAG=y + +# TinyUSB vendor (WebUSB) + CDC (Web Serial) class drivers for the framed +# protocol; HID count only so usb_device's HID code paths compile. +CONFIG_TINYUSB_VENDOR_COUNT=1 +CONFIG_TINYUSB_CDC_ENABLED=y +CONFIG_TINYUSB_CDC_COUNT=1 +CONFIG_TINYUSB_HID_COUNT=1 +CONFIG_TINYUSB_VENDOR_RX_BUFSIZE=4096 +CONFIG_TINYUSB_VENDOR_TX_BUFSIZE=4096 +CONFIG_TINYUSB_CDC_TX_BUFSIZE=4096 + +# Task statistics for MonitorService (GET_TASKS / TASKS stream): the FreeRTOS +# trace facility + run-time stats, and the core id per task. +CONFIG_FREERTOS_USE_TRACE_FACILITY=y +CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS=y +CONFIG_FREERTOS_VTASKLIST_INCLUDE_COREID=y + +CONFIG_COMPILER_CXX_EXCEPTIONS=y diff --git a/components/system/idf_component.yml b/components/system/idf_component.yml new file mode 100644 index 0000000000..5929b2bd8f --- /dev/null +++ b/components/system/idf_component.yml @@ -0,0 +1,28 @@ +## IDF Component Manager Manifest File +license: "MIT" +description: "System identity / status (chip, app, partitions, reset reason, uptime, heap) and restart control (reboot, reboot into the ROM bootloader), plus a framed stream service and a browser console (WebUSB / Web Serial)" +url: "https://github.com/esp-cpp/espp/tree/main/components/system" +repository: "https://github.com/esp-cpp/espp.git" +maintainers: + - William Emfinger +documentation: "https://esp-cpp.github.io/espp/system/system.html" +examples: + - path: example +tags: + - cpp + - Component + - System + - Reboot + - Bootloader + - Info + - WebUSB + - WebSerial + - USB +dependencies: + idf: + # esp_app_get_description() and esp_clk_tree.h first appear in ESP-IDF v5.1 + version: '>=5.1' + espp/base_component: '>=1.0' + espp/format: '>=1.0' + espp/stream_frame: '>=1.0' + espp/dispatcher: '>=1.0' diff --git a/components/system/include/detail/system_protocol.hpp b/components/system/include/detail/system_protocol.hpp new file mode 100644 index 0000000000..d98cff5b04 --- /dev/null +++ b/components/system/include/detail/system_protocol.hpp @@ -0,0 +1,350 @@ +#pragma once + +// Wire protocol of espp::SystemService: device identity / status plus reboot +// control over the espp stream_frame codec, routed by an espp::Dispatcher on +// module 7 by default (`espp.system` v1 through discovery). +// +// This header is deliberately host-buildable (stream_frame.hpp + the standard +// library only) so the codec is unit-tested on the host +// (components/system/test/system_host_test.cpp) and so host tools can reuse +// it. All multi-byte fields are little-endian. +// +// Requests (host -> device, reply flag clear): +// 0x01 GET_INFO (no payload) +// 0x02 REBOOT [delay_ms u16] +// 0x03 REBOOT_TO_BOOTLOADER [delay_ms u16] +// Replies (device -> host, high bit set = frame reply flag): +// 0x81 INFO a list of tagged records [tag u8][len u8][value...]; a host +// skips tags it does not know, so fields can be added without a +// version bump (see InfoTag for the values). +// 0x83 OK [request_type u8] +// 0x84 ERROR [request_type u8][code u32][utf8 message] +// A reboot request is acknowledged with OK first; the device restarts after +// the requested delay (clamped to at least Config::min_restart_delay). + +#include +#include +#include +#include +#include +#include +#include + +#include "stream_frame.hpp" + +namespace espp::detail::system_protocol { + +/// Default dispatcher module id (a routing key only; see SystemService::Config::module). +inline constexpr uint8_t kModule = 7; +/// Stable protocol identifier + version advertised through discovery. +inline constexpr const char *kProtocol = "espp.system"; +inline constexpr uint16_t kProtocolVersion = 1; + +/// Frame `type` values within the system module. +enum class Type : uint8_t { + // host -> device + GetInfo = 0x01, + Reboot = 0x02, + RebootToBootloader = 0x03, + // device -> host (high bit set) + Info = 0x81, + Ok = 0x83, + Error = 0x84, +}; + +/// Tags of the INFO records. Values are little-endian; `str` is raw UTF-8 +/// (the record's len is the string length). +enum class InfoTag : uint8_t { + ChipModel = 1, ///< str, e.g. "ESP32-S3" + ChipRevision = 2, ///< u16 (MXX: major * 100 + minor) + Cores = 3, ///< u8 + ChipFeatures = 4, ///< u32 (CHIP_FEATURE_* bitmask) + IdfVersion = 5, ///< str + ProjectName = 6, ///< str + AppVersion = 7, ///< str + BuildDate = 8, ///< str + BuildTime = 9, ///< str + ElfSha256 = 10, ///< 32 raw bytes + RunningPartition = 11, ///< str (partition label) + BootPartition = 12, ///< str (partition label) + OtaState = 13, ///< u8 (esp_ota_img_states_t; 0xFF = undefined / not an OTA partition) + ResetReason = 14, ///< u8 (esp_reset_reason_t) + UptimeMs = 15, ///< u64 + Mac = 16, ///< 6 raw bytes (base MAC) + FlashSize = 17, ///< u32 bytes + PsramSize = 18, ///< u32 bytes (0 = none) + CpuMhz = 19, ///< u32 + FreeHeap = 20, ///< u32 bytes + MinFreeHeap = 21, ///< u32 bytes + Capabilities = 22, ///< u32 (kCapReboot | kCapBootloader) +}; + +/// Capabilities bits (InfoTag::Capabilities). +inline constexpr uint32_t kCapReboot = 0x01; ///< REBOOT is allowed by the service. +inline constexpr uint32_t kCapBootloader = 0x02; ///< REBOOT_TO_BOOTLOADER is allowed AND supported. + +/// Whether a type value is a device->host reply. +inline constexpr bool is_reply(Type type) { return (static_cast(type) & 0x80) != 0; } + +/// Build an encoded frame for a system message (device->host types map to the +/// frame reply flag). +inline std::vector build_frame(Type type, std::span payload = {}, + uint8_t module = kModule) { + return espp::stream_frame::build_frame(is_reply(type), module, static_cast(type), + payload); +} + +// ---- INFO record encoding ------------------------------------------------------ + +/// Builds an INFO payload one tagged record at a time. Values longer than 255 +/// bytes are truncated (strings) -- every value defined today is far shorter. +class InfoBuilder { +public: + InfoBuilder &str(InfoTag tag, std::string_view s) { + const size_t n = s.size() > 255 ? 255 : s.size(); + header(tag, n); + out_.insert(out_.end(), s.begin(), s.begin() + static_cast(n)); + return *this; + } + InfoBuilder &u8(InfoTag tag, uint8_t v) { + header(tag, 1); + out_.push_back(v); + return *this; + } + InfoBuilder &u16(InfoTag tag, uint16_t v) { + header(tag, 2); + espp::stream_frame::put_u16(out_, v); + return *this; + } + InfoBuilder &u32(InfoTag tag, uint32_t v) { + header(tag, 4); + espp::stream_frame::put_u32(out_, v); + return *this; + } + InfoBuilder &u64(InfoTag tag, uint64_t v) { + header(tag, 8); + espp::stream_frame::put_u32(out_, static_cast(v)); + espp::stream_frame::put_u32(out_, static_cast(v >> 32)); + return *this; + } + InfoBuilder &bytes(InfoTag tag, std::span b) { + const size_t n = b.size() > 255 ? 255 : b.size(); + header(tag, n); + out_.insert(out_.end(), b.begin(), b.begin() + static_cast(n)); + return *this; + } + const std::vector &payload() const { return out_; } + std::vector take() { return std::move(out_); } + +private: + void header(InfoTag tag, size_t len) { + out_.push_back(static_cast(tag)); + out_.push_back(static_cast(len)); + } + std::vector out_; +}; + +/// A decoded INFO payload: every field is optional (absent when the device did +/// not send the tag). Unknown tags are skipped, so a newer device decodes fine. +struct Info { + std::optional chip_model; + std::optional chip_revision; + std::optional cores; + std::optional chip_features; + std::optional idf_version; + std::optional project_name; + std::optional app_version; + std::optional build_date; + std::optional build_time; + std::optional> elf_sha256; + std::optional running_partition; + std::optional boot_partition; + std::optional ota_state; + std::optional reset_reason; + std::optional uptime_ms; + std::optional> mac; + std::optional flash_size; + std::optional psram_size; + std::optional cpu_mhz; + std::optional free_heap; + std::optional min_free_heap; + std::optional capabilities; + size_t unknown_tags{0}; ///< how many records carried a tag this decoder does not know +}; + +/// Decode an INFO payload. Returns nullopt only on a truncated record (a record +/// whose declared length runs past the payload); a record with an unexpected +/// length for a known fixed-size tag is skipped (counted as unknown). +inline std::optional decode_info(std::span p) { + Info info; + size_t i = 0; + while (i < p.size()) { + if (i + 2 > p.size()) + return std::nullopt; + const uint8_t tag = p[i]; + const size_t len = p[i + 1]; + i += 2; + if (i + len > p.size()) + return std::nullopt; + const std::span v = p.subspan(i, len); + i += len; + auto as_str = [&]() { return std::string(reinterpret_cast(v.data()), v.size()); }; + auto u8 = [&](std::optional &dst) { + if (len == 1) + dst = v[0]; + else + ++info.unknown_tags; + }; + auto u16 = [&](std::optional &dst) { + if (len == 2) + dst = espp::stream_frame::get_u16(v); + else + ++info.unknown_tags; + }; + auto u32 = [&](std::optional &dst) { + if (len == 4) + dst = espp::stream_frame::get_u32(v); + else + ++info.unknown_tags; + }; + switch (static_cast(tag)) { + case InfoTag::ChipModel: + info.chip_model = as_str(); + break; + case InfoTag::ChipRevision: + u16(info.chip_revision); + break; + case InfoTag::Cores: + u8(info.cores); + break; + case InfoTag::ChipFeatures: + u32(info.chip_features); + break; + case InfoTag::IdfVersion: + info.idf_version = as_str(); + break; + case InfoTag::ProjectName: + info.project_name = as_str(); + break; + case InfoTag::AppVersion: + info.app_version = as_str(); + break; + case InfoTag::BuildDate: + info.build_date = as_str(); + break; + case InfoTag::BuildTime: + info.build_time = as_str(); + break; + case InfoTag::ElfSha256: + if (len == 32) { + std::array sha{}; + std::copy(v.begin(), v.end(), sha.begin()); + info.elf_sha256 = sha; + } else { + ++info.unknown_tags; + } + break; + case InfoTag::RunningPartition: + info.running_partition = as_str(); + break; + case InfoTag::BootPartition: + info.boot_partition = as_str(); + break; + case InfoTag::OtaState: + u8(info.ota_state); + break; + case InfoTag::ResetReason: + u8(info.reset_reason); + break; + case InfoTag::UptimeMs: + if (len == 8) { + info.uptime_ms = static_cast(espp::stream_frame::get_u32(v)) | + (static_cast(espp::stream_frame::get_u32(v.subspan(4))) << 32); + } else { + ++info.unknown_tags; + } + break; + case InfoTag::Mac: + if (len == 6) { + std::array mac{}; + std::copy(v.begin(), v.end(), mac.begin()); + info.mac = mac; + } else { + ++info.unknown_tags; + } + break; + case InfoTag::FlashSize: + u32(info.flash_size); + break; + case InfoTag::PsramSize: + u32(info.psram_size); + break; + case InfoTag::CpuMhz: + u32(info.cpu_mhz); + break; + case InfoTag::FreeHeap: + u32(info.free_heap); + break; + case InfoTag::MinFreeHeap: + u32(info.min_free_heap); + break; + case InfoTag::Capabilities: + u32(info.capabilities); + break; + default: + ++info.unknown_tags; + break; + } + } + return info; +} + +// ---- request / reply helpers --------------------------------------------------- + +/// Encode a REBOOT / REBOOT_TO_BOOTLOADER payload. +inline std::vector encode_delay(uint16_t delay_ms) { + std::vector p; + espp::stream_frame::put_u16(p, delay_ms); + return p; +} + +/// Decode a REBOOT / REBOOT_TO_BOOTLOADER payload (an empty payload means 0 ms). +inline std::optional decode_delay(std::span p) { + if (p.empty()) + return 0; + if (p.size() < 2) + return std::nullopt; + return espp::stream_frame::get_u16(p); +} + +/// Encode an OK payload. +inline std::vector encode_ok(uint8_t request_type) { return {request_type}; } + +/// Encode an ERROR payload. +inline std::vector encode_error(uint8_t request_type, uint32_t code, + std::string_view message) { + std::vector p; + p.push_back(request_type); + espp::stream_frame::put_u32(p, code); + p.insert(p.end(), message.begin(), message.end()); + return p; +} + +/// Decoded ERROR payload. +struct Error { + uint8_t request_type{0}; + uint32_t code{0}; + std::string message; +}; + +inline std::optional decode_error(std::span p) { + if (p.size() < 5) + return std::nullopt; + Error e; + e.request_type = p[0]; + e.code = espp::stream_frame::get_u32(p.subspan(1)); + e.message.assign(reinterpret_cast(p.data() + 5), p.size() - 5); + return e; +} + +} // namespace espp::detail::system_protocol diff --git a/components/system/include/system_control.hpp b/components/system/include/system_control.hpp new file mode 100644 index 0000000000..d251830a40 --- /dev/null +++ b/components/system/include/system_control.hpp @@ -0,0 +1,134 @@ +#pragma once + +#include +#include +#include + +#include "sdkconfig.h" + +#include "esp_system.h" +#include "soc/soc.h" + +// The "force download boot" bit lives in a different always-on register on +// each chip family; ESP-IDF's own ROM USB console (esp_usb_cdc_rom_console) +// sets it the same way before restarting. +#if CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3 || CONFIG_IDF_TARGET_ESP32C3 || \ + CONFIG_IDF_TARGET_ESP32C2 +#include "soc/rtc_cntl_reg.h" +#define ESPP_SYSTEM_DOWNLOAD_BOOT_RTC_CNTL 1 +#elif CONFIG_IDF_TARGET_ESP32C6 || CONFIG_IDF_TARGET_ESP32H2 || CONFIG_IDF_TARGET_ESP32C5 || \ + CONFIG_IDF_TARGET_ESP32C61 || CONFIG_IDF_TARGET_ESP32H21 +#include "soc/lp_aon_reg.h" +#define ESPP_SYSTEM_DOWNLOAD_BOOT_LP_AON 1 +#elif CONFIG_IDF_TARGET_ESP32P4 +#include "soc/lp_system_reg.h" +#define ESPP_SYSTEM_DOWNLOAD_BOOT_LP_SYSTEM 1 +#endif +#if CONFIG_IDF_TARGET_ESP32S2 +#include "esp32s2/rom/usb/chip_usb_dw_wrapper.h" +#include "esp32s2/rom/usb/usb_persist.h" +#define ESPP_SYSTEM_USB_PERSIST 1 +#elif CONFIG_IDF_TARGET_ESP32S3 +#include "esp32s3/rom/usb/chip_usb_dw_wrapper.h" +#include "esp32s3/rom/usb/usb_persist.h" +#define ESPP_SYSTEM_USB_PERSIST 1 +#endif + +namespace espp { + +/** + * @brief Restart control: a plain reboot, and a reboot into the ROM + * bootloader's download (serial flashing) mode. + * + * reboot_to_bootloader() sets the chip's "force download boot" flag in its + * always-on register and restarts, so the next boot stays in the ROM + * download mode instead of running the app -- exactly what holding the BOOT + * strap during a reset does, without touching a button. The device then + * re-enumerates as the ROM's own flashing interface: the USB CDC / DFU device + * on the ESP32-S2 / -S3 native USB port (the ROM's USB stack is kept + * persistent across the reset), or USB-Serial-JTAG on the ESP32-C3 / -C6 / + * -H2 / -C5 / -C61 / -H21 / -P4 -- so `esptool` / `idf.py flash` can program + * it. On the classic ESP32 there is no software path (only the GPIO0 strap): + * bootloader_reboot_supported() is false and reboot_to_bootloader() fails + * with operation_not_supported. + * + * Both functions restart immediately; use the delayed variants (or + * espp::SystemService, which replies before restarting) when a reply must + * leave the transport first. + */ +class SystemControl { +public: + /// @brief Whether reboot_to_bootloader() is implemented for this chip. + static constexpr bool bootloader_reboot_supported() { +#if ESPP_SYSTEM_DOWNLOAD_BOOT_RTC_CNTL || ESPP_SYSTEM_DOWNLOAD_BOOT_LP_AON || \ + ESPP_SYSTEM_DOWNLOAD_BOOT_LP_SYSTEM + return true; +#else + return false; +#endif + } + + /// @brief Restart the chip (esp_restart()); does not return. + [[noreturn]] static void reboot() { esp_restart(); } + + /// @brief Restart into the ROM bootloader's download mode. + /// @param ec Set to operation_not_supported on chips without a software path + /// (classic ESP32); then returns false without restarting. + /// @return Does not return on success; false on failure. + static bool reboot_to_bootloader(std::error_code &ec) { + ec.clear(); + if (!bootloader_reboot_supported()) { + ec = std::make_error_code(std::errc::operation_not_supported); + return false; + } + arm_download_boot(); + esp_restart(); + return true; // not reached + } + + /// @brief Restart after a delay, from a detached thread, so the caller can + /// finish (e.g. send a reply) first. Returns immediately. + static void reboot_after(std::chrono::milliseconds delay) { + std::thread([delay]() { + std::this_thread::sleep_for(delay); + esp_restart(); + }).detach(); + } + + /// @brief Restart into download mode after a delay, from a detached thread. + /// Returns immediately; false (nothing scheduled) if unsupported. + static bool reboot_to_bootloader_after(std::chrono::milliseconds delay, std::error_code &ec) { + ec.clear(); + if (!bootloader_reboot_supported()) { + ec = std::make_error_code(std::errc::operation_not_supported); + return false; + } + std::thread([delay]() { + std::this_thread::sleep_for(delay); + arm_download_boot(); + esp_restart(); + }).detach(); + return true; + } + +private: + /// Set the chip's force-download-boot flag (survives the reset that follows). + static void arm_download_boot() { +#if ESPP_SYSTEM_USB_PERSIST + // keep the ROM's USB device attached across the reset so the host sees the + // download-mode CDC / DFU interface without a full re-plug + chip_usb_set_persist_flags(USBDC_PERSIST_ENA); +#endif +#if ESPP_SYSTEM_DOWNLOAD_BOOT_RTC_CNTL + REG_WRITE(RTC_CNTL_OPTION1_REG, RTC_CNTL_FORCE_DOWNLOAD_BOOT); +#elif ESPP_SYSTEM_DOWNLOAD_BOOT_LP_AON + // a 1-bit flag on most chips; a 2-bit field on the C5 where 1 = "force + // download boot 0 (UART / USB)" -- writing the field value 1 covers both + REG_SET_FIELD(LP_AON_SYS_CFG_REG, LP_AON_FORCE_DOWNLOAD_BOOT, 1); +#elif ESPP_SYSTEM_DOWNLOAD_BOOT_LP_SYSTEM + REG_SET_FIELD(LP_SYSTEM_REG_SYS_CTRL_REG, LP_SYSTEM_REG_FORCE_DOWNLOAD_BOOT, 1); +#endif + } +}; + +} // namespace espp diff --git a/components/system/include/system_info.hpp b/components/system/include/system_info.hpp new file mode 100644 index 0000000000..04c2074abd --- /dev/null +++ b/components/system/include/system_info.hpp @@ -0,0 +1,297 @@ +#pragma once + +#include +#include +#include +#include + +#include "sdkconfig.h" + +#include "esp_app_desc.h" +#include "esp_chip_info.h" +#include "esp_clk_tree.h" +#include "esp_flash.h" +#include "esp_idf_version.h" +#include "esp_mac.h" +#include "esp_ota_ops.h" +#include "esp_system.h" +#include "esp_timer.h" +#if CONFIG_SPIRAM +#include "esp_psram.h" +#endif + +#include "format.hpp" + +namespace espp { + +/** + * @brief Static accessors for the identity and status of the running system: + * the chip, the ESP-IDF version, the application description (project + * name, version, build date / time, ELF SHA-256), the running / boot + * partitions and OTA state, the reset reason, uptime, base MAC, flash + * and PSRAM sizes, CPU frequency and heap figures. + * + * Everything is a thin, allocation-light wrapper over the corresponding + * ESP-IDF call so it can be used from any task. collect() gathers it all into + * one Snapshot (what espp::SystemService reports to a host) and to_string() + * renders a human-readable summary for logs. + * + * \section system_info_ex1 SystemInfo Example + * \snippet system_example.cpp system_example + */ +class SystemInfo { +public: + /// Everything collect() gathers. + struct Snapshot { + std::string chip_model; ///< e.g. "ESP32-S3" + uint16_t chip_revision{0}; ///< MXX: major * 100 + minor + uint8_t cores{0}; ///< CPU core count + uint32_t chip_features{0}; ///< CHIP_FEATURE_* bitmask + std::string idf_version; ///< e.g. "v6.1" + std::string project_name; ///< CMake project name + std::string app_version; ///< PROJECT_VER + std::string build_date; ///< compile date + std::string build_time; ///< compile time + std::array elf_sha256{}; ///< SHA-256 of the application ELF + std::string running_partition; ///< label of the partition the app runs from + std::string boot_partition; ///< label of the partition the bootloader will boot next + uint8_t ota_state{0xFF}; ///< esp_ota_img_states_t of the running partition; 0xFF = n/a + uint8_t reset_reason{0}; ///< esp_reset_reason_t + uint64_t uptime_ms{0}; ///< time since boot + std::array mac{}; ///< base MAC address + uint32_t flash_size{0}; ///< bytes + uint32_t psram_size{0}; ///< bytes (0 = none / disabled) + uint32_t cpu_mhz{0}; ///< current CPU frequency + uint32_t free_heap{0}; ///< bytes + uint32_t min_free_heap{0}; ///< bytes, lowest since boot + }; + + /// @brief Chip model name for an esp_chip_model_t ("ESP32-S3", ...). + static const char *chip_model_name(esp_chip_model_t model) { + // Compared by value rather than by enumerator so this compiles against + // ESP-IDF releases that predate the newer chips. + switch (static_cast(model)) { + case 1: + return "ESP32"; + case 2: + return "ESP32-S2"; + case 9: + return "ESP32-S3"; + case 5: + return "ESP32-C3"; + case 12: + return "ESP32-C2"; + case 13: + return "ESP32-C6"; + case 16: + return "ESP32-H2"; + case 18: + return "ESP32-P4"; + case 20: + return "ESP32-C61"; + case 23: + return "ESP32-C5"; + case 25: + return "ESP32-H21"; + case 28: + return "ESP32-H4"; + case 32: + return "ESP32-S31"; + default: + return "ESP32 (unknown model)"; + } + } + + /// @brief Human-readable name for an esp_reset_reason_t. + static const char *reset_reason_name(esp_reset_reason_t reason) { + switch (static_cast(reason)) { + case 0: + return "unknown"; + case 1: + return "power-on"; + case 2: + return "external pin"; + case 3: + return "software (esp_restart)"; + case 4: + return "panic"; + case 5: + return "interrupt watchdog"; + case 6: + return "task watchdog"; + case 7: + return "other watchdog"; + case 8: + return "deep-sleep wake"; + case 9: + return "brownout"; + case 10: + return "SDIO"; + case 11: + return "USB"; + case 12: + return "JTAG"; + case 13: + return "efuse error"; + case 14: + return "power glitch"; + case 15: + return "CPU lockup"; + default: + return "unknown"; + } + } + + /// @brief Human-readable name for an OTA image state byte (Snapshot::ota_state). + static const char *ota_state_name(uint8_t state) { + switch (state) { + case 0: + return "new"; + case 1: + return "pending verify"; + case 2: + return "valid"; + case 3: + return "invalid"; + case 4: + return "aborted"; + case 0xFF: + return "n/a"; + default: + return "undefined"; + } + } + + /// @brief Chip information (model, revision, cores, features). + static esp_chip_info_t chip_info() { + esp_chip_info_t info{}; + esp_chip_info(&info); + return info; + } + + /// @brief The ESP-IDF version string the app was built with. + static std::string idf_version() { return esp_get_idf_version(); } + + /// @brief The application description embedded in the running image. + static const esp_app_desc_t &app_description() { return *esp_app_get_description(); } + + /// @brief Label of the partition the running app was loaded from ("" if unknown). + static std::string running_partition() { + const esp_partition_t *p = esp_ota_get_running_partition(); + return p ? p->label : ""; + } + + /// @brief Label of the partition the bootloader will boot next ("" if unknown). + static std::string boot_partition() { + const esp_partition_t *p = esp_ota_get_boot_partition(); + return p ? p->label : ""; + } + + /// @brief OTA image state of the running partition (esp_ota_img_states_t as a + /// byte; 0xFF when the app runs from a factory partition or the state + /// cannot be read). + static uint8_t ota_state() { + const esp_partition_t *p = esp_ota_get_running_partition(); + esp_ota_img_states_t state; + if (!p || esp_ota_get_state_partition(p, &state) != ESP_OK) + return 0xFF; + return state == ESP_OTA_IMG_UNDEFINED ? 0xFE : static_cast(state); + } + + /// @brief Why the chip last reset. + static esp_reset_reason_t reset_reason() { return esp_reset_reason(); } + + /// @brief Milliseconds since boot. + static uint64_t uptime_ms() { return static_cast(esp_timer_get_time() / 1000); } + + /// @brief The chip's base (factory-programmed) MAC address. + static std::array base_mac() { + std::array mac{}; + esp_efuse_mac_get_default(mac.data()); + return mac; + } + + /// @brief Size of the main SPI flash in bytes (0 if it cannot be read). + static uint32_t flash_size() { + uint32_t size = 0; + if (esp_flash_get_size(nullptr, &size) != ESP_OK) + return 0; + return size; + } + + /// @brief Size of the PSRAM in bytes (0 without PSRAM / with CONFIG_SPIRAM off). + static uint32_t psram_size() { +#if CONFIG_SPIRAM + return static_cast(esp_psram_get_size()); +#else + return 0; +#endif + } + + /// @brief Current CPU frequency in MHz. + static uint32_t cpu_mhz() { + uint32_t hz = 0; + if (esp_clk_tree_src_get_freq_hz(SOC_MOD_CLK_CPU, ESP_CLK_TREE_SRC_FREQ_PRECISION_CACHED, + &hz) != ESP_OK || + hz == 0) + return CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ; + return hz / 1000000u; + } + + /// @brief Free heap in bytes (default capabilities). + static uint32_t free_heap() { return esp_get_free_heap_size(); } + + /// @brief Lowest free heap since boot, in bytes. + static uint32_t min_free_heap() { return esp_get_minimum_free_heap_size(); } + + /// @brief Gather everything into one Snapshot. + static Snapshot collect() { + Snapshot s; + const esp_chip_info_t chip = chip_info(); + s.chip_model = chip_model_name(chip.model); + s.chip_revision = chip.revision; + s.cores = chip.cores; + s.chip_features = chip.features; + s.idf_version = idf_version(); + const esp_app_desc_t &app = app_description(); + s.project_name = + std::string(app.project_name, strnlen(app.project_name, sizeof(app.project_name))); + s.app_version = std::string(app.version, strnlen(app.version, sizeof(app.version))); + s.build_date = std::string(app.date, strnlen(app.date, sizeof(app.date))); + s.build_time = std::string(app.time, strnlen(app.time, sizeof(app.time))); + std::memcpy(s.elf_sha256.data(), app.app_elf_sha256, s.elf_sha256.size()); + s.running_partition = running_partition(); + s.boot_partition = boot_partition(); + s.ota_state = ota_state(); + s.reset_reason = static_cast(reset_reason()); + s.uptime_ms = uptime_ms(); + s.mac = base_mac(); + s.flash_size = flash_size(); + s.psram_size = psram_size(); + s.cpu_mhz = cpu_mhz(); + s.free_heap = free_heap(); + s.min_free_heap = min_free_heap(); + return s; + } + + /// @brief A multi-line human-readable summary of a Snapshot. + static std::string to_string(const Snapshot &s) { + return fmt::format( + "{} rev {}.{} ({} core{}), ESP-IDF {}\n" + "app: {} {} built {} {}\n" + "partition: running '{}', boot '{}', OTA state {}\n" + "reset: {}; uptime {} ms; MAC {:02x}:{:02x}:{:02x}:{:02x}:{:02x}:{:02x}\n" + "flash {} KiB, PSRAM {} KiB, CPU {} MHz, heap free {} (min {})", + s.chip_model, s.chip_revision / 100, s.chip_revision % 100, s.cores, + s.cores == 1 ? "" : "s", s.idf_version, s.project_name, s.app_version, s.build_date, + s.build_time, s.running_partition, s.boot_partition, ota_state_name(s.ota_state), + reset_reason_name(static_cast(s.reset_reason)), s.uptime_ms, s.mac[0], + s.mac[1], s.mac[2], s.mac[3], s.mac[4], s.mac[5], s.flash_size / 1024, s.psram_size / 1024, + s.cpu_mhz, s.free_heap, s.min_free_heap); + } + + /// @brief Collect and format in one call (for a boot banner). + static std::string to_string() { return to_string(collect()); } +}; + +} // namespace espp diff --git a/components/system/include/system_service.hpp b/components/system/include/system_service.hpp new file mode 100644 index 0000000000..7c3da30722 --- /dev/null +++ b/components/system/include/system_service.hpp @@ -0,0 +1,284 @@ +#pragma once + +// espp::SystemService -- device identity / status and reboot control as a +// transport-agnostic dispatcher module (detail/system_protocol.hpp is the wire +// spec). It follows the same contract as espp::OtaService / CoreDumpService: +// requests are handled and the reply built under an internal mutex, the +// `send` callback always runs after that mutex is released, and frames for +// other modules / reply-flagged frames are ignored so the service coexists +// with other protocols on one stream. +// +// Wiring (one line per transport): +// +// espp::SystemService system_service({.send = [&](auto f) { usb.write_vendor(f); }}); +// dispatcher.register_module(system_service); // module 7 + discovery metadata + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "dispatcher.hpp" +#include "stream_frame.hpp" + +#include "base_component.hpp" +#include "detail/system_protocol.hpp" +#include "system_control.hpp" +#include "system_info.hpp" + +namespace espp { + +/** + * @brief Serves espp::SystemInfo and espp::SystemControl over any framed byte + * stream (dispatcher module 7 by default; see Config::module). + * + * GET_INFO answers with a tagged-record snapshot (see + * detail/system_protocol.hpp; hosts skip tags they do not know). REBOOT and + * REBOOT_TO_BOOTLOADER reply OK first and restart after the requested delay + * (at least Config::min_restart_delay, so the reply leaves the transport) + * from a detached thread -- the same pattern as OtaService's post-update + * restart. Both are guarded: Config::allow_reboot / allow_bootloader switch + * them off (ERROR "not permitted"), the optional Config::on_reboot_request + * callback can veto a specific request (an application with a motor running + * can refuse or defer), and REBOOT_TO_BOOTLOADER is refused with "not + * supported" on chips without a software download-mode path (classic ESP32). + * The INFO capabilities record tells a host up front which of the two it may + * offer. + * + * **Threading**: an internal mutex covers the parser and request handling; + * the `send` callback and the veto callback run after it is released. Drive + * one instance from one context per byte stream (a Dispatcher / + * DispatcherWorker, or a single task calling feed()). + * + * \section system_service_ex1 SystemService Example + * \snippet system_example.cpp system_example + */ +class SystemService : public BaseComponent { +public: + using Stream = espp::stream_frame::StreamParser; + using Type = espp::detail::system_protocol::Type; + + /// Default dispatcher module id (7). Only a routing key: Config::module serves + /// on any id, and the hosted system console finds it through discovery (by + /// kProtocol). + static constexpr uint8_t kModule = espp::detail::system_protocol::kModule; + /// Stable protocol identifier + version advertised through discovery. + static constexpr const char *kProtocol = espp::detail::system_protocol::kProtocol; + static constexpr uint16_t kProtocolVersion = espp::detail::system_protocol::kProtocolVersion; + + /// Which restart a host asked for (passed to Config::on_reboot_request). + enum class RebootKind : uint8_t { + Reboot, ///< plain restart + Bootloader, ///< restart into the ROM download mode + }; + + /// Transmits one encoded reply frame to the host. + using send_fn = std::function frame)>; + /// Asked (outside the lock) before a permitted reboot is acknowledged; + /// return false to veto it (the host gets ERROR "refused by the application"). + using reboot_request_fn = std::function; + + /// Configuration for the SystemService. + struct Config { + send_fn send{nullptr}; ///< Transmits an encoded reply frame (required). + /// Dispatcher module id this instance answers on (and stamps on its + /// replies). A routing key only: hosts find whichever id is chosen through + /// discovery (by kProtocol), so any id 0x00..0xEF is fine. + uint8_t module{kModule}; + bool allow_reboot{true}; ///< Serve REBOOT (else ERROR "not permitted"). + bool allow_bootloader{true}; ///< Serve REBOOT_TO_BOOTLOADER (else ERROR "not permitted"). + /// Optional veto for a specific reboot request; called after the allow_* + /// checks, outside the lock. nullptr = every permitted request proceeds. + reboot_request_fn on_reboot_request{nullptr}; + /// Lower bound on the delay between the OK reply and the restart, so the + /// reply reaches the host even when it asked for 0 ms. + std::chrono::milliseconds min_restart_delay{250}; + espp::Logger::Verbosity log_level{espp::Logger::Verbosity::WARN}; ///< Logger verbosity. + }; + + /// @brief Construct the service. + explicit SystemService(const Config &config) + : BaseComponent("SystemService", config.log_level) + , config_(config) {} + + /// @brief The dispatcher module id this service answers on (Config::module). + uint8_t module_id() const { return config_.module; } + + /// @brief Discovery metadata for registering this service on a Dispatcher. + Dispatcher::ModuleInfo module_info() const { + return {.name = "System", + .app = "system_console.html", + .description = "Device info, reboot and bootloader entry", + .protocol = kProtocol, + .protocol_version = kProtocolVersion}; + } + + /// @brief The capabilities bitmask GET_INFO reports (kCapReboot / kCapBootloader). + uint32_t capabilities() const { + namespace proto = espp::detail::system_protocol; + uint32_t caps = 0; + if (config_.allow_reboot) + caps |= proto::kCapReboot; + if (config_.allow_bootloader && SystemControl::bootloader_reboot_supported()) + caps |= proto::kCapBootloader; + return caps; + } + + /// @brief Build the INFO payload (a SystemInfo snapshot as tagged records). + std::vector build_info() const { + namespace proto = espp::detail::system_protocol; + const SystemInfo::Snapshot s = SystemInfo::collect(); + proto::InfoBuilder b; + b.str(proto::InfoTag::ChipModel, s.chip_model) + .u16(proto::InfoTag::ChipRevision, s.chip_revision) + .u8(proto::InfoTag::Cores, s.cores) + .u32(proto::InfoTag::ChipFeatures, s.chip_features) + .str(proto::InfoTag::IdfVersion, s.idf_version) + .str(proto::InfoTag::ProjectName, s.project_name) + .str(proto::InfoTag::AppVersion, s.app_version) + .str(proto::InfoTag::BuildDate, s.build_date) + .str(proto::InfoTag::BuildTime, s.build_time) + .bytes(proto::InfoTag::ElfSha256, s.elf_sha256) + .str(proto::InfoTag::RunningPartition, s.running_partition) + .str(proto::InfoTag::BootPartition, s.boot_partition) + .u8(proto::InfoTag::OtaState, s.ota_state) + .u8(proto::InfoTag::ResetReason, s.reset_reason) + .u64(proto::InfoTag::UptimeMs, s.uptime_ms) + .bytes(proto::InfoTag::Mac, s.mac) + .u32(proto::InfoTag::FlashSize, s.flash_size) + .u32(proto::InfoTag::PsramSize, s.psram_size) + .u32(proto::InfoTag::CpuMhz, s.cpu_mhz) + .u32(proto::InfoTag::FreeHeap, s.free_heap) + .u32(proto::InfoTag::MinFreeHeap, s.min_free_heap) + .u32(proto::InfoTag::Capabilities, capabilities()); + return b.take(); + } + + /** + * @brief Dispatcher entry point: handle one routed frame. Frames for other + * modules and reply-flagged frames are ignored, so this can be + * registered directly: `dispatcher.register_module(service)`. + */ + void handle(const espp::stream_frame::Frame &frame) { + if (frame.module != module_id() || frame.is_reply()) + return; + handle_frame(frame.type, frame.payload); + } + + /// @brief Feed received transport bytes (standalone use, without a Dispatcher). + void feed(std::span data) { + std::vector frames; + { + std::lock_guard lock(mutex_); + frames = parser_.feed(data); + } + for (const auto &frame : frames) + handle(frame); + } + + /// @brief Discard any partially-buffered frame bytes (standalone feed() use). + void reset_parser() { + std::lock_guard lock(mutex_); + parser_.reset(); + } + + /** + * @brief Handle one already-parsed request frame. + * @return true if the type belongs to the system protocol (a reply was + * sent), false if it was ignored. + * @note The `send` and veto callbacks run after the internal mutex is released. + */ + bool handle_frame(uint8_t type, std::span payload) { + namespace proto = espp::detail::system_protocol; + std::error_code ec; + switch (static_cast(type)) { + case Type::GetInfo: { + std::vector info; + { + std::lock_guard lock(mutex_); + info = build_info(); + } + send(proto::build_frame(Type::Info, info, module_id())); + return true; + } + case Type::Reboot: + case Type::RebootToBootloader: { + const bool bootloader = static_cast(type) == Type::RebootToBootloader; + const auto delay = proto::decode_delay(payload); + if (!delay) { + send_error(type, std::errc::invalid_argument, "malformed request (expected u16 delay_ms)"); + return true; + } + const bool allowed = bootloader ? config_.allow_bootloader : config_.allow_reboot; + if (!allowed) { + send_error(type, std::errc::operation_not_permitted, + bootloader ? "reboot into bootloader is disabled on this device" + : "reboot is disabled on this device"); + return true; + } + if (bootloader && !SystemControl::bootloader_reboot_supported()) { + send_error(type, std::errc::operation_not_supported, + "this chip has no software path into download mode (use the BOOT strap)"); + return true; + } + // the veto runs outside the lock: the application may take its own locks + if (config_.on_reboot_request && + !config_.on_reboot_request(bootloader ? RebootKind::Bootloader : RebootKind::Reboot)) { + send_error(type, std::errc::operation_canceled, + "refused by the application (try again later)"); + return true; + } + const auto wait = std::max(std::chrono::milliseconds(*delay), config_.min_restart_delay); + // reply first, then restart from a detached thread so the reply leaves + // the transport and the caller's task (the transport worker) is never blocked + send(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id())); + logger_.info("{} in {} ms", bootloader ? "rebooting into the bootloader" : "rebooting", + wait.count()); + if (bootloader) + SystemControl::reboot_to_bootloader_after(wait, ec); + else + SystemControl::reboot_after(wait); + return true; + } + default: + // not a system request: ignore so the service can share a stream + return false; + } + } + +protected: + /// Transmit an encoded frame. Must be called WITHOUT the mutex held. + void send(const std::vector &frame) { + if (frame.empty()) + return; + if (!config_.send) { + logger_.warn("no send function configured; dropping a {}-byte reply", frame.size()); + return; + } + config_.send(frame); + } + + void send_error(uint8_t request_type, std::errc errc, std::string_view message) { + namespace proto = espp::detail::system_protocol; + logger_.warn("{} (type 0x{:02x})", message, request_type); + send(proto::build_frame(Type::Error, + proto::encode_error(request_type, static_cast(errc), message), + module_id())); + } + +private: + Config config_; + mutable std::mutex mutex_; + Stream parser_; +}; + +// Compile-time check that the service keeps satisfying the dispatcher's module +// contract (module_id() / module_info() / handle(frame)). +static_assert(DispatcherModuleConcept); + +} // namespace espp diff --git a/components/system/test/system_host_test.cpp b/components/system/test/system_host_test.cpp new file mode 100644 index 0000000000..017ebd4e03 --- /dev/null +++ b/components/system/test/system_host_test.cpp @@ -0,0 +1,170 @@ +// Host-buildable unit tests for the SystemService wire codec +// (include/detail/system_protocol.hpp). Build & run with: +// c++ -std=c++20 -I../include -I../../stream_frame/include system_host_test.cpp -o test && ./test +// +// The codec needs no ESP-IDF headers; these are golden encode/decode tests so +// the browser console and any host tool can rely on the byte layout. + +#include +#include +#include +#include +#include +#include + +#include "detail/system_protocol.hpp" + +namespace sp = espp::detail::system_protocol; +namespace sf = espp::stream_frame; + +static int g_failures = 0; +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::printf(" FAIL: %s (line %d)\n", #cond, __LINE__); \ + ++g_failures; \ + } \ + } while (0) + +static void test_info_golden() { + std::printf("test_info_golden\n"); + sp::InfoBuilder b; + b.str(sp::InfoTag::ChipModel, "ESP32-S3") + .u16(sp::InfoTag::ChipRevision, 2) + .u8(sp::InfoTag::Cores, 2); + const auto &p = b.payload(); + // [1][8]"ESP32-S3" [2][2][0x02 0x00] [3][1][2] + const uint8_t golden[] = {1, 8, 'E', 'S', 'P', '3', '2', '-', 'S', + '3', 2, 2, 0x02, 0x00, 3, 1, 2}; + CHECK(p.size() == sizeof(golden) && std::equal(golden, golden + sizeof(golden), p.begin())); + const auto info = sp::decode_info(p); + CHECK(info && info->chip_model == "ESP32-S3" && info->chip_revision == 2 && info->cores == 2); + CHECK(info && !info->idf_version && info->unknown_tags == 0); +} + +static void test_info_all_tags() { + std::printf("test_info_all_tags\n"); + std::array sha{}; + for (size_t i = 0; i < sha.size(); ++i) + sha[i] = static_cast(i); + const std::array mac = {0x24, 0x6F, 0x28, 0x01, 0x02, 0x03}; + sp::InfoBuilder b; + b.str(sp::InfoTag::ChipModel, "ESP32-P4") + .u16(sp::InfoTag::ChipRevision, 100) + .u8(sp::InfoTag::Cores, 2) + .u32(sp::InfoTag::ChipFeatures, 0x81) + .str(sp::InfoTag::IdfVersion, "v6.1") + .str(sp::InfoTag::ProjectName, "system_example") + .str(sp::InfoTag::AppVersion, "1.2.3") + .str(sp::InfoTag::BuildDate, "Sep 30 2026") + .str(sp::InfoTag::BuildTime, "12:34:56") + .bytes(sp::InfoTag::ElfSha256, sha) + .str(sp::InfoTag::RunningPartition, "ota_0") + .str(sp::InfoTag::BootPartition, "ota_1") + .u8(sp::InfoTag::OtaState, 2) + .u8(sp::InfoTag::ResetReason, 3) + .u64(sp::InfoTag::UptimeMs, 0x0000000123456789ULL) + .bytes(sp::InfoTag::Mac, mac) + .u32(sp::InfoTag::FlashSize, 16 * 1024 * 1024) + .u32(sp::InfoTag::PsramSize, 8 * 1024 * 1024) + .u32(sp::InfoTag::CpuMhz, 360) + .u32(sp::InfoTag::FreeHeap, 123456) + .u32(sp::InfoTag::MinFreeHeap, 100000) + .u32(sp::InfoTag::Capabilities, sp::kCapReboot | sp::kCapBootloader); + const auto info = sp::decode_info(b.payload()); + CHECK(info); + if (!info) + return; + CHECK(info->chip_model == "ESP32-P4" && info->chip_revision == 100 && info->cores == 2 && + info->chip_features == 0x81u); + CHECK(info->idf_version == "v6.1" && info->project_name == "system_example" && + info->app_version == "1.2.3" && info->build_date == "Sep 30 2026" && + info->build_time == "12:34:56"); + CHECK(info->elf_sha256 && (*info->elf_sha256)[31] == 31); + CHECK(info->running_partition == "ota_0" && info->boot_partition == "ota_1" && + info->ota_state == 2 && info->reset_reason == 3); + CHECK(info->uptime_ms == 0x0000000123456789ULL); + CHECK(info->mac && (*info->mac)[0] == 0x24 && (*info->mac)[5] == 0x03); + CHECK(info->flash_size == 16u * 1024 * 1024 && info->psram_size == 8u * 1024 * 1024 && + info->cpu_mhz == 360); + CHECK(info->free_heap == 123456 && info->min_free_heap == 100000); + CHECK(info->capabilities == (sp::kCapReboot | sp::kCapBootloader)); + CHECK(info->unknown_tags == 0); + // the u64 is little-endian on the wire: find the record and check its bytes + const auto &p = b.payload(); + bool found = false; + for (size_t i = 0; i + 1 < p.size();) { + const uint8_t tag = p[i], len = p[i + 1]; + if (tag == static_cast(sp::InfoTag::UptimeMs)) { + CHECK(len == 8 && p[i + 2] == 0x89 && p[i + 3] == 0x67 && p[i + 5] == 0x23 && p[i + 9] == 0); + found = true; + } + i += 2 + len; + } + CHECK(found); +} + +static void test_info_unknown_and_truncated() { + std::printf("test_info_unknown_and_truncated\n"); + // an unknown tag (200) with a 3-byte value is skipped; the record after it decodes + std::vector p = {200, 3, 0xAA, 0xBB, 0xCC, 3, 1, 1}; + auto info = sp::decode_info(p); + CHECK(info && info->cores == 1 && info->unknown_tags == 1); + // a known fixed-size tag with the wrong length is skipped, not misdecoded + p = {3, 2, 1, 1, 19, 4, 0x68, 0x01, 0, 0}; + info = sp::decode_info(p); + CHECK(info && !info->cores && info->cpu_mhz == 360 && info->unknown_tags == 1); + // a record whose length runs past the payload is rejected + p = {1, 8, 'E', 'S', 'P'}; + CHECK(!sp::decode_info(p)); + p = {1}; + CHECK(!sp::decode_info(p)); + // an empty payload decodes to "nothing known" + info = sp::decode_info({}); + CHECK(info && !info->chip_model && info->unknown_tags == 0); + // a string longer than 255 bytes is truncated by the builder, not corrupted + sp::InfoBuilder b; + b.str(sp::InfoTag::ProjectName, std::string(300, 'p')); + info = sp::decode_info(b.payload()); + CHECK(info && info->project_name && info->project_name->size() == 255); +} + +static void test_requests_and_replies() { + std::printf("test_requests_and_replies\n"); + const auto d = sp::encode_delay(750); + CHECK(d.size() == 2 && d[0] == 0xEE && d[1] == 0x02); + CHECK(sp::decode_delay(d) == 750); + CHECK(sp::decode_delay({}) == 0); // empty payload = no delay + const uint8_t one[] = {1}; + CHECK(!sp::decode_delay(one)); + CHECK(sp::encode_ok(0x02) == std::vector{0x02}); + const auto e = sp::encode_error(0x03, 95, "not supported"); + CHECK(e.size() == 5 + 13 && e[0] == 3 && e[1] == 95 && e[2] == 0 && e[5] == 'n'); + const auto de = sp::decode_error(e); + CHECK(de && de->request_type == 3 && de->code == 95 && de->message == "not supported"); + CHECK(!sp::decode_error(std::span(e.data(), 4))); + // frames: replies carry the reply flag, requests do not; module is stamped + const auto req = sp::build_frame(sp::Type::Reboot, d, 7); + CHECK(req[2] == 0x10 && req[3] == 7 && req[4] == 0x02); + const auto rep = sp::build_frame(sp::Type::Info, {}, 11); + CHECK(rep[2] == 0x11 && rep[3] == 11 && rep[4] == 0x81); + sf::StreamParser parser; + std::vector stream(req); + stream.insert(stream.end(), rep.begin(), rep.end()); + const auto frames = parser.feed(stream); + CHECK(frames.size() == 2 && !frames[0].is_reply() && frames[0].payload.size() == 2 && + frames[1].is_reply() && frames[1].module == 11); +} + +int main() { + test_info_golden(); + test_info_all_tags(); + test_info_unknown_and_truncated(); + test_requests_and_replies(); + if (g_failures) { + std::printf("%d FAILURE(S)\n", g_failures); + return 1; + } + std::printf("ALL TESTS PASSED\n"); + return 0; +} diff --git a/components/system/web/system_console.html b/components/system/web/system_console.html new file mode 100644 index 0000000000..1050c920f3 --- /dev/null +++ b/components/system/web/system_console.html @@ -0,0 +1,1361 @@ + + + + + + espp System Console (WebUSB / Web Serial) + + + + + +
+
+

espp System Console (WebUSB / Web Serial)

+ Disconnected +
+ +
+ This browser supports neither WebUSB nor Web Serial. Use a Chromium-based + browser (Chrome / Edge / Opera) on a secure origin (https, localhost or file://). +
+ +
+

Device

+
+ + + +
+

+ Not connected. WebUSB talks to the vendor (0xFF) interface (default filter VID 0x1209); + Web Serial talks to the CDC port, where the device console and the protocol share one + stream — this page renders the console text and speaks the protocol at the same time. +

+
+ +
+

System info

+
+ + +
+
+
+ +
+

Control

+
+ + + + +
+
+
+
+ + +
+
+
+ + + +
+

Log

+
+ + +
+
+
+ +
+ Speaks the espp system (module 7, espp.system) and monitor + (module 8, espp.monitor) services over stream_frame v2 framing (both ids are discovered on + connect). Everything runs locally in your browser. +
+
+ + + + diff --git a/doc/Doxyfile b/doc/Doxyfile index 24e15bb869..5924a853bc 100755 --- a/doc/Doxyfile +++ b/doc/Doxyfile @@ -185,6 +185,7 @@ EXAMPLE_PATH = \ $(PROJECT_PATH)/components/st25dv/example/main/st25dv_example.cpp \ $(PROJECT_PATH)/components/st7123touch/example/main/st7123touch_example.cpp \ $(PROJECT_PATH)/components/state_machine/example/main/hfsm_example.cpp \ + $(PROJECT_PATH)/components/system/example/main/system_example.cpp \ $(PROJECT_PATH)/components/stream_frame/example/main/stream_frame_example.cpp \ $(PROJECT_PATH)/components/switch_pro/example/main/switch_pro_example.cpp \ $(PROJECT_PATH)/components/sx126x/example/main/sx126x_example.cpp \ @@ -387,6 +388,7 @@ INPUT = \ $(PROJECT_PATH)/components/matouch-rotary-display/include/matouch-rotary-display.hpp \ $(PROJECT_PATH)/components/max1704x/include/max1704x.hpp \ $(PROJECT_PATH)/components/monitor/include/heap_monitor.hpp \ + $(PROJECT_PATH)/components/monitor/include/monitor_service.hpp \ $(PROJECT_PATH)/components/monitor/include/task_monitor.hpp \ $(PROJECT_PATH)/components/motor_controller/include/basicmicro_commands.hpp \ $(PROJECT_PATH)/components/motor_controller/include/motor_controller.hpp \ @@ -471,6 +473,9 @@ INPUT = \ $(PROJECT_PATH)/components/tabulate/include/tabulate.hpp \ $(PROJECT_PATH)/components/task/include/task.hpp \ $(PROJECT_PATH)/components/task/include/run_on_core.hpp \ + $(PROJECT_PATH)/components/system/include/system_control.hpp \ + $(PROJECT_PATH)/components/system/include/system_info.hpp \ + $(PROJECT_PATH)/components/system/include/system_service.hpp \ $(PROJECT_PATH)/components/telemetry/include/telemetry.hpp \ $(PROJECT_PATH)/components/thermistor/include/thermistor.hpp \ $(PROJECT_PATH)/components/thread_pool/include/qos_band.hpp \ diff --git a/doc/en/core/monitor.rst b/doc/en/core/monitor.rst index df39dac650..aaed1af913 100644 --- a/doc/en/core/monitor.rst +++ b/doc/en/core/monitor.rst @@ -64,3 +64,29 @@ Task Monitor API Reference -------------------------- .. include-build-file:: inc/task_monitor.inc + +Monitor Service +--------------- + +The `MonitorService` class serves the heap-region and task statistics above +over **any byte stream** as a :doc:`dispatcher <../dispatcher/dispatcher>` +module (``espp.monitor`` v1, module id 8 by default; ``Config::module`` moves +an instance and hosts find it through discovery by its protocol id). +``GET_HEAP`` answers with one record per configured heap region +(``Config::heap_regions``, ``MALLOC_CAP_*`` masks; regions the chip does not +have are left out), ``GET_TASKS`` with the ``TaskMonitor`` table (name, CPU %, +stack high-water mark, priority, core — it needs +``CONFIG_FREERTOS_USE_TRACE_FACILITY`` and +``CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS``, else the list is empty), and +``SET_STREAM`` starts a task that sends either or both periodically so a host +can plot them live. The wire codec (``detail/monitor_protocol.hpp``) is +host-buildable and unit-tested (``test/monitor_host_test.cpp``). The hosted +`espp System Console `_ +web app renders the heap gauges and a live task table; see the +:doc:`system <../system/system>` component's example, which exposes both +services over USB. + +Monitor Service API Reference +----------------------------- + +.. include-build-file:: inc/monitor_service.inc diff --git a/doc/en/dispatcher/custom_modules.rst b/doc/en/dispatcher/custom_modules.rst index 30bc7fb3ec..070cbc8882 100644 --- a/doc/en/dispatcher/custom_modules.rst +++ b/doc/en/dispatcher/custom_modules.rst @@ -66,6 +66,8 @@ Module id Protocol Protocol id 4 Crash dump (``espp::CoreDumpService``) ``espp.coredump`` v1 5 CAN bridge (``components/canopen``) ``espp.can-bridge`` v1 6 MCP266 motor-controller console (``espp::Mcp266Service``) ``espp.mcp266`` v1 +7 System info / reboot control (``espp::SystemService``) ``espp.system`` v1 +8 Heap / task monitor (``espp::MonitorService``) ``espp.monitor`` v1 0xF0-0xFE reserved for dispatcher / meta use 0xFF capability discovery ========= ======================================================== ================================== @@ -738,7 +740,7 @@ module's hosted web app (see `Hosting your webapp`_ below); the hub treats it as a same-directory relative link. `protocol` is how a host *identifies* your module regardless of the id it is registered on: give your protocol a stable, namespaced id (espp's are ``espp.ota``, ``espp.coredump``, ``espp.telemetry``, -``espp.mcp266``, ``espp.can-bridge``, ``espp.haptics``, +``espp.mcp266``, ``espp.can-bridge``, ``espp.haptics``, ``espp.system``, ``espp.monitor``, ``espp.coredump-crash-trigger``) and bump `protocol_version` when the wire format changes; a host reports a version other than the one it implements as a warning. Both are new in discovery payload version 2 and optional. diff --git a/doc/en/dispatcher/dispatcher.rst b/doc/en/dispatcher/dispatcher.rst index 75148e80e6..50d77a03b2 100644 --- a/doc/en/dispatcher/dispatcher.rst +++ b/doc/en/dispatcher/dispatcher.rst @@ -28,6 +28,8 @@ Module id Protocol Protocol id (discover 4 crash dump ``espp.coredump`` v1 5 CAN bridge ``espp.can-bridge`` v1 6 MCP266 console ``espp.mcp266`` v1 +7 System info / reboot ``espp.system`` v1 +8 Heap / task monitor ``espp.monitor`` v1 0xF0-0xFE reserved (meta) 0xFF capability discovery ========= ============================================== =============================== diff --git a/doc/en/index.rst b/doc/en/index.rst index 0ac3b24fdd..15cc46a6a6 100755 --- a/doc/en/index.rst +++ b/doc/en/index.rst @@ -23,6 +23,7 @@ collected under :doc:`web_apps`. core/index coredump/index + system/index .. toctree:: :maxdepth: 1 diff --git a/doc/en/system/index.rst b/doc/en/system/index.rst new file mode 100644 index 0000000000..5b05413856 --- /dev/null +++ b/doc/en/system/index.rst @@ -0,0 +1,16 @@ +System APIs +*********** + +.. toctree:: + :maxdepth: 1 + + system + +The `System` component reports what a device is and how it is doing — +chip, ESP-IDF version, application description, partitions and OTA state, +reset reason, uptime, MAC, memory sizes, CPU frequency and heap — and +controls restarts (a plain reboot, or a reboot into the ROM bootloader's +download mode), as plain C++ APIs and as a transport-agnostic stream service +with a browser web app (WebUSB / Web Serial). The :doc:`monitor +<../core/monitor>` component's ``MonitorService`` complements it with live +heap and task statistics. diff --git a/doc/en/system/system.rst b/doc/en/system/system.rst new file mode 100644 index 0000000000..a05223a6b2 --- /dev/null +++ b/doc/en/system/system.rst @@ -0,0 +1,61 @@ +System Info, Control & Service +****************************** + +The `SystemInfo` class is a set of static getters over the corresponding +ESP-IDF calls: the chip model, revision, core count and feature flags, the +ESP-IDF version, the application description embedded in the image (project +name, version, build date and time, ELF SHA-256), the running and boot +partitions with the OTA image state, the reset reason, uptime, base MAC, flash +and PSRAM sizes, CPU frequency and the free / lowest-free heap. +``collect()`` gathers everything into one ``Snapshot`` and ``to_string()`` +renders a boot-banner style summary. + +The `SystemControl` class restarts the device: ``reboot()`` (``esp_restart``), +and ``reboot_to_bootloader()`` which sets the chip's *force download boot* +flag in its always-on register and restarts, so the next boot stays in the +ROM download mode instead of running the app — what holding the BOOT strap +during a reset does, without a button. The device then re-enumerates as the +ROM's own flashing interface (the USB CDC / DFU device on the ESP32-S2 / -S3 +native USB port, kept attached across the reset; USB-Serial-JTAG on the +ESP32-C3 / -C6 / -H2 / -C5 / -C61 / -H21 / -P4), ready for ``esptool`` / +``idf.py flash``. The classic ESP32 has no software path (only the GPIO0 +strap): ``bootloader_reboot_supported()`` is false there and the call fails +with ``operation_not_supported``. The ``*_after(delay)`` variants restart from +a detached thread so a reply can leave the transport first. + +The `SystemService` class serves both over **any byte stream** as a +:doc:`dispatcher <../dispatcher/dispatcher>` module (``espp.system`` v1, +module id 7 by default; ``Config::module`` moves an instance and hosts find +it through discovery by its protocol id). ``GET_INFO`` answers with a list of +tagged records (``[tag u8][len u8][value]``) a host decodes while skipping +tags it does not know, so fields can be added without a version bump. +``REBOOT`` and ``REBOOT_TO_BOOTLOADER`` reply ``OK`` first and restart after +the requested delay (clamped to ``Config::min_restart_delay``). Both are +guarded: ``Config::allow_reboot`` / ``allow_bootloader`` switch them off, the +optional ``on_reboot_request`` callback can veto a specific request (an +application with a motor running can refuse or defer), and the bootloader +restart is refused on chips without a software path. The ``INFO`` +capabilities record tells a host up front which of the two it may offer. + +The hosted `espp System Console +`_ web app speaks +the protocol over **WebUSB** (vendor interface) or **Web Serial** (CDC, where +it doubles as a serial monitor): a device-info panel, the two restart buttons +(with an in-page confirmation), and — when the device also advertises the +:doc:`monitor <../core/monitor>` component's ``MonitorService`` — heap-region +gauges and a live, sortable task table with a stream toggle. + +.. ------------------------------- Example ------------------------------------- + +.. toctree:: + + system_example + +.. ---------------------------- API Reference ---------------------------------- + +API Reference +------------- + +.. include-build-file:: inc/system_info.inc +.. include-build-file:: inc/system_control.inc +.. include-build-file:: inc/system_service.inc diff --git a/doc/en/system/system_example.md b/doc/en/system/system_example.md new file mode 100644 index 0000000000..2842ab6627 --- /dev/null +++ b/doc/en/system/system_example.md @@ -0,0 +1,42 @@ +# System Example + +[![Badge](https://components.espressif.com/components/espp/system/badge.svg)](https://components.espressif.com/components/espp/system) + +This example shows how to use the `espp::SystemService` and +`espp::MonitorService` components to expose device info / control and live +heap + task statistics over the native USB port (WebUSB + Web Serial) of an +ESP32-S3, for the hosted system console web app. + +## How to use example + +### Hardware Required + +An ESP32-S3 board with its native USB port connected to the host. + +### Build and Flash + +Build the project and flash it to the board, then run monitor tool to view serial output: + +``` +idf.py -p PORT flash monitor +``` + +(Replace PORT with the name of the serial port to use.) + +(To exit the serial monitor, type ``Ctrl-]``.) + +See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects. + +## Example Output + +``` +I (317) System Example: Starting system info + control example +I (327) System Example: System: +ESP32-S3 rev 0.2 (2 cores), ESP-IDF v6.1 +app: system_example 1 built Sep 30 2026 12:34:56 +partition: running 'factory', boot 'factory', OTA state n/a +reset: power-on; uptime 320 ms; MAC 34:85:18:xx:xx:xx +flash 8192 KiB, PSRAM 0 KiB, CPU 240 MHz, heap free 318412 (min 318412) +I (357) System Example: Reboot into the bootloader is supported on this chip +I (1077) System Example: Ready. Connect the native USB port and open the system console ... +``` diff --git a/doc/en/web_apps.rst b/doc/en/web_apps.rst index 6e5cc9c56d..f04dc6856a 100644 --- a/doc/en/web_apps.rst +++ b/doc/en/web_apps.rst @@ -42,6 +42,11 @@ device may serve a protocol on any dispatcher module id. over SDO). - **MCP266 Console** (``mcp266_console.html``) — status, motor, and configuration controls for the :doc:`mcp266 ` motor controller. +- **System Console** (``system_console.html``) — device info (chip, firmware, + partitions, reset reason, uptime, memory), reboot and reboot-into-bootloader + for the :doc:`system ` component, plus live heap gauges and a + task table when the device serves the :doc:`monitor ` + component's ``MonitorService``. Motor control ============= From dcbf8bfb473ecb081f72682abcefddbba74db208 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 10:25:51 -0400 Subject: [PATCH 2/7] fix(monitor): build the TASKS reply per stats Kconfig instead of suppressing cppcheck CI's cppcheck evaluates the configuration where the FreeRTOS stats are enabled, so the two inline suppressions were unmatched (and fatal). The conversion now lives under the stats Kconfig and the other configuration returns an empty list directly: no suppressions in either. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/monitor/include/monitor_service.hpp | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/components/monitor/include/monitor_service.hpp b/components/monitor/include/monitor_service.hpp index a1d12f8538..f3b0def61c 100644 --- a/components/monitor/include/monitor_service.hpp +++ b/components/monitor/include/monitor_service.hpp @@ -217,16 +217,10 @@ class MonitorService : public BaseComponent { /// The TASKS payload (capped at the frame payload limit). std::vector build_tasks() { namespace proto = espp::detail::monitor_protocol; +#if CONFIG_FREERTOS_USE_TRACE_FACILITY && CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS const auto infos = TaskMonitor::get_latest_info_vector(); -#if !(CONFIG_FREERTOS_USE_TRACE_FACILITY && CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS) - logger_.warn_rate_limited("task statistics need CONFIG_FREERTOS_USE_TRACE_FACILITY and " - "CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS; reporting no tasks"); -#endif std::vector tasks; tasks.reserve(infos.size()); - // (without the FreeRTOS stats Kconfig `infos` is provably empty; the - // conversion is still the right code for the configured build) - // cppcheck-suppress knownEmptyContainer std::transform( infos.begin(), infos.end(), std::back_inserter(tasks), [](const TaskMonitor::TaskInfo &t) { return proto::TaskEntry{ @@ -238,11 +232,17 @@ class MonitorService : public BaseComponent { }); size_t encoded = 0; auto payload = proto::encode_tasks(tasks, espp::stream_frame::kMaxPayloadSize, &encoded); - // cppcheck-suppress unsignedLessThanZero if (encoded < tasks.size()) logger_.warn_rate_limited("TASKS payload full: reporting {} of {} tasks", encoded, tasks.size()); return payload; +#else + // TaskMonitor cannot collect anything without the FreeRTOS stats Kconfig: + // report an empty list (and say why, once in a while) + logger_.warn_rate_limited("task statistics need CONFIG_FREERTOS_USE_TRACE_FACILITY and " + "CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS; reporting no tasks"); + return proto::encode_tasks({}, espp::stream_frame::kMaxPayloadSize, nullptr); +#endif } void start_stream(std::chrono::milliseconds period, uint8_t what) { From 932203b0f0518e2c2c339d42a7702ceaf2df2748 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 11:01:26 -0400 Subject: [PATCH 3/7] fix(system,monitor): review round 1 (hub ?module= for the monitor, per-transport send mutex, frame cap, console state) - Console: ?module=N is applied to whichever ident module N advertises the protocol of (the monitor's for espp.monitor, else the system's), so the Device Hub's Monitor link works; the monitor panel counts as present only on a protocol match (both services share the app); capabilities, info, heap and task state are reset on every new connection; heap-region labels use the real MALLOC_CAP_* bits; the task-table sort controls are buttons with aria-sort on the active column. - Example: one send mutex per transport that both services' `send` go through, so a streamed monitor event never interleaves a system reply. - MonitorService::Config::max_frame_bytes (4096): the TASKS payload is capped so header + payload + CRC fits one write; test + docs. - system_protocol.hpp: tag 13 documents both sentinels (0xFE undefined, 0xFF unavailable / not an OTA partition). - CI / docs housekeeping: system in upload_components.yml, build matrix entry and Doxyfile entries in alphabetical order after sx126x, system_example.md includes the example README. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- .github/workflows/build.yml | 6 +- .github/workflows/upload_components.yml | 1 + components/monitor/README.md | 3 +- .../include/detail/monitor_protocol.hpp | 3 +- .../monitor/include/monitor_service.hpp | 19 ++++- components/monitor/test/monitor_host_test.cpp | 13 ++++ .../system/example/main/system_example.cpp | 19 ++++- .../system/include/detail/system_protocol.hpp | 21 +++--- components/system/web/system_console.html | 75 ++++++++++++++----- doc/Doxyfile | 8 +- doc/en/core/monitor.rst | 4 +- doc/en/system/system_example.md | 42 +---------- 12 files changed, 129 insertions(+), 85 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 5d22020fd4..c52b793ecd 100755 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -130,9 +130,6 @@ jobs: - path: 'components/coredump/example' target: esp32s3 command: 'IDF_COMPONENT_MANAGER=0 idf.py build' - - path: 'components/system/example' - target: esp32s3 - command: 'IDF_COMPONENT_MANAGER=0 idf.py build' - path: 'components/cst816/example' target: esp32s3 - path: 'components/csv/example' @@ -331,6 +328,9 @@ jobs: target: esp32s3 - path: 'components/sx126x/example' target: esp32s3 + - path: 'components/system/example' + target: esp32s3 + command: 'IDF_COMPONENT_MANAGER=0 idf.py build' - path: 'components/t-deck/example' target: esp32s3 - path: 'components/t-dongle-s3/example' diff --git a/.github/workflows/upload_components.yml b/.github/workflows/upload_components.yml index 9dc15ca287..c106b1f356 100755 --- a/.github/workflows/upload_components.yml +++ b/.github/workflows/upload_components.yml @@ -172,6 +172,7 @@ jobs: components/state_machine components/switch_pro components/sx126x + components/system components/t_keyboard components/t-deck components/t-dongle-s3 diff --git a/components/monitor/README.md b/components/monitor/README.md index 8c19212e0c..b4e2a94009 100644 --- a/components/monitor/README.md +++ b/components/monitor/README.md @@ -31,7 +31,8 @@ or into a table for visualization. statistics over any framed byte stream as an `espp::Dispatcher` module (`espp.monitor` v1, module 8 by default): `GET_HEAP` (one record per configured `MALLOC_CAP_*` region), `GET_TASKS` (the `TaskMonitor` table; needs -`CONFIG_FREERTOS_USE_TRACE_FACILITY` + `CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS`) +`CONFIG_FREERTOS_USE_TRACE_FACILITY` + `CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS`; +capped so the whole frame fits `Config::max_frame_bytes`, 4096 by default) and `SET_STREAM` (periodic HEAP / TASKS events). The wire codec lives in `include/detail/monitor_protocol.hpp` (host-buildable, tested by `test/monitor_host_test.cpp`). The hosted diff --git a/components/monitor/include/detail/monitor_protocol.hpp b/components/monitor/include/detail/monitor_protocol.hpp index 24603efb31..4c34f26f95 100644 --- a/components/monitor/include/detail/monitor_protocol.hpp +++ b/components/monitor/include/detail/monitor_protocol.hpp @@ -24,7 +24,8 @@ // 0x84 ERROR [request_type u8][code u32][utf8 message] // HEAP / TASKS answer the matching GET_* request and are also sent // unsolicited while streaming is enabled (same encoding, so a host decodes -// both the same way). A TASKS payload is capped at the frame payload limit: +// both the same way). A TASKS payload is capped (MonitorService: to fit +// Config::max_frame_bytes with the frame overhead; at most the payload limit): // tasks that would not fit are dropped from the END of the list. #include diff --git a/components/monitor/include/monitor_service.hpp b/components/monitor/include/monitor_service.hpp index f3b0def61c..aed6bfb82b 100644 --- a/components/monitor/include/monitor_service.hpp +++ b/components/monitor/include/monitor_service.hpp @@ -87,6 +87,12 @@ class MonitorService : public BaseComponent { std::vector heap_regions{MALLOC_CAP_DEFAULT, MALLOC_CAP_INTERNAL, MALLOC_CAP_SPIRAM}; /// Shortest streaming period a host may request (SET_STREAM is clamped to it). std::chrono::milliseconds min_stream_period{100}; + /// Largest encoded frame (header + payload + CRC) `send` can carry in one + /// write: the TASKS payload is capped so the whole frame fits (tasks that + /// do not fit are dropped from the end, logged). 4096 matches the default + /// TinyUSB vendor / CDC TX FIFO of the espp examples; the stream_frame + /// maximum is kMaxFrameSize (4111). + size_t max_frame_bytes{4096}; /// The streaming task (started on the first SET_STREAM enable). Task::BaseConfig task_config{.name = "monitor_stream", .stack_size_bytes = 6 * 1024}; espp::Logger::Verbosity log_level{espp::Logger::Verbosity::WARN}; ///< Logger verbosity. @@ -231,7 +237,7 @@ class MonitorService : public BaseComponent { .core_id = static_cast(t.core_id)}; }); size_t encoded = 0; - auto payload = proto::encode_tasks(tasks, espp::stream_frame::kMaxPayloadSize, &encoded); + auto payload = proto::encode_tasks(tasks, max_tasks_payload(), &encoded); if (encoded < tasks.size()) logger_.warn_rate_limited("TASKS payload full: reporting {} of {} tasks", encoded, tasks.size()); @@ -241,10 +247,19 @@ class MonitorService : public BaseComponent { // report an empty list (and say why, once in a while) logger_.warn_rate_limited("task statistics need CONFIG_FREERTOS_USE_TRACE_FACILITY and " "CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS; reporting no tasks"); - return proto::encode_tasks({}, espp::stream_frame::kMaxPayloadSize, nullptr); + return proto::encode_tasks({}, max_tasks_payload(), nullptr); #endif } + /// The TASKS payload cap: Config::max_frame_bytes less the frame overhead + /// (a 9-byte header, no correlation id, plus the CRC), never above the codec's + /// own payload limit. + size_t max_tasks_payload() const { + constexpr size_t overhead = espp::stream_frame::kHeaderSize + espp::stream_frame::kCrcSize; + const size_t cap = config_.max_frame_bytes > overhead ? config_.max_frame_bytes - overhead : 0; + return std::min(cap, espp::stream_frame::kMaxPayloadSize); + } + void start_stream(std::chrono::milliseconds period, uint8_t what) { std::lock_guard lock(mutex_); period_.store(std::max(period, config_.min_stream_period)); diff --git a/components/monitor/test/monitor_host_test.cpp b/components/monitor/test/monitor_host_test.cpp index 3e760e100b..09743c855f 100644 --- a/components/monitor/test/monitor_host_test.cpp +++ b/components/monitor/test/monitor_host_test.cpp @@ -88,6 +88,19 @@ static void test_tasks_roundtrip_and_cap() { size_t n2 = 0; const auto capped = mp::encode_tasks(tasks, 1 + mp::task_entry_size("main") + 3, &n2); CHECK(n2 == 1 && capped[0] == 1 && capped.size() == 1 + mp::task_entry_size("main")); + // the service caps the payload so the whole frame (9-byte header + CRC) + // fits its max_frame_bytes: with the default 4096 that is a 4083-byte payload + const size_t service_cap = 4096 - (sf::kHeaderSize + sf::kCrcSize); + CHECK(service_cap == 4083); + size_t n4 = 0; + std::vector lots(400, {.name = std::string(16, 'y'), + .cpu_percent = 1, + .high_water_mark = 1, + .priority = 1, + .core_id = 0}); + const auto fitted = mp::encode_tasks(lots, service_cap, &n4); + CHECK(fitted.size() <= service_cap && n4 == fitted[0] && n4 < 400); + CHECK(mp::build_frame(mp::Type::Tasks, fitted).size() <= 4096); // a 4096-byte payload never overflows: 500 tasks with long names std::vector many(500, {.name = std::string(40, 'x'), .cpu_percent = 1, diff --git a/components/system/example/main/system_example.cpp b/components/system/example/main/system_example.cpp index a2340d9030..ba502ae794 100644 --- a/components/system/example/main/system_example.cpp +++ b/components/system/example/main/system_example.cpp @@ -1,4 +1,5 @@ #include +#include #include #include @@ -53,10 +54,20 @@ extern "C" void app_main(void) { espp::UsbDevice usb(usb_cfg); // Replies go back on the stream the request came in on: one send function - // per transport. Both services on one transport share it, and each service - // serializes its own frames; the USB writes are all-or-nothing per call. - auto vendor_send = [&](std::span frame) { usb.write_vendor(frame); }; - auto cdc_send = [&](std::span frame) { usb.write_cdc(frame); }; + // per transport. Both services share each transport and only serialize + // their OWN frames (a streamed monitor event and a system reply come from + // different tasks), so every device->host write on a transport goes through + // one application-level mutex; write_vendor / write_cdc are all-or-nothing + // per call, so a frame is never truncated or interleaved. + std::mutex vendor_tx_mutex, cdc_tx_mutex; + auto vendor_send = [&](std::span frame) { + std::lock_guard lock(vendor_tx_mutex); + usb.write_vendor(frame); + }; + auto cdc_send = [&](std::span frame) { + std::lock_guard lock(cdc_tx_mutex); + usb.write_cdc(frame); + }; // The application decides whether a reboot may happen right now: this demo // permits every request and logs it. A real application would refuse (or diff --git a/components/system/include/detail/system_protocol.hpp b/components/system/include/detail/system_protocol.hpp index d98cff5b04..81add2c19e 100644 --- a/components/system/include/detail/system_protocol.hpp +++ b/components/system/include/detail/system_protocol.hpp @@ -67,16 +67,17 @@ enum class InfoTag : uint8_t { ElfSha256 = 10, ///< 32 raw bytes RunningPartition = 11, ///< str (partition label) BootPartition = 12, ///< str (partition label) - OtaState = 13, ///< u8 (esp_ota_img_states_t; 0xFF = undefined / not an OTA partition) - ResetReason = 14, ///< u8 (esp_reset_reason_t) - UptimeMs = 15, ///< u64 - Mac = 16, ///< 6 raw bytes (base MAC) - FlashSize = 17, ///< u32 bytes - PsramSize = 18, ///< u32 bytes (0 = none) - CpuMhz = 19, ///< u32 - FreeHeap = 20, ///< u32 bytes - MinFreeHeap = 21, ///< u32 bytes - Capabilities = 22, ///< u32 (kCapReboot | kCapBootloader) + OtaState = 13, ///< u8 esp_ota_img_states_t (0 new .. 4 aborted); 0xFE = ESP_OTA_IMG_UNDEFINED, + ///< 0xFF = state unavailable / not an OTA partition (SystemInfo::ota_state()) + ResetReason = 14, ///< u8 (esp_reset_reason_t) + UptimeMs = 15, ///< u64 + Mac = 16, ///< 6 raw bytes (base MAC) + FlashSize = 17, ///< u32 bytes + PsramSize = 18, ///< u32 bytes (0 = none) + CpuMhz = 19, ///< u32 + FreeHeap = 20, ///< u32 bytes + MinFreeHeap = 21, ///< u32 bytes + Capabilities = 22, ///< u32 (kCapReboot | kCapBootloader) }; /// Capabilities bits (InfoTag::Capabilities). diff --git a/components/system/web/system_console.html b/components/system/web/system_console.html index 1050c920f3..c12e61ab08 100644 --- a/components/system/web/system_console.html +++ b/components/system/web/system_console.html @@ -168,7 +168,12 @@ table.tasks { border-collapse: collapse; width: 100%; font-size: 12.5px; margin-top: 8px; } table.tasks th, table.tasks td { text-align: left; padding: 4px 10px 4px 0; border-bottom: 1px solid var(--panel-border); white-space: nowrap; font-variant-numeric: tabular-nums; } - table.tasks th { color: var(--text-muted); font-weight: 600; cursor: pointer; user-select: none; } + table.tasks th { color: var(--text-muted); font-weight: 600; } + table.tasks th button.sort { font: inherit; font-weight: 600; color: inherit; background: none; border: 0; padding: 0; cursor: pointer; } + table.tasks th button.sort:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } + table.tasks th[aria-sort="ascending"] button.sort::after { content: " \25B4"; } + table.tasks th[aria-sort="descending"] button.sort::after { content: " \25BE"; } + .sr-only { position: absolute; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); white-space: nowrap; } table.tasks td.num, table.tasks th.num { text-align: right; padding-right: 14px; } table.tasks td.bar { width: 120px; } .cpubar { height: 8px; border-radius: 999px; background: var(--chip-bg); border: 1px solid var(--panel-border); overflow: hidden; } @@ -271,12 +276,12 @@

Heap & tasks (monitor module)

- - - - - - + + + + + +
@@ -359,9 +364,12 @@

Log

const CHIP_FEATURES = [[0x01, "embedded flash"], [0x02, "Wi-Fi 2.4 GHz"], [0x10, "BLE"], [0x20, "BT classic"], [0x40, "IEEE 802.15.4"], [0x80, "embedded PSRAM"]]; // MALLOC_CAP_* names for the heap gauges (esp_heap_caps.h) - const HEAP_CAPS = [[1 << 10, "internal"], [1 << 11, "SPIRAM"], [1 << 12, "invalid"], [1 << 13, "retention"], - [1 << 14, "RTC fast"], [1 << 15, "TCM"], [1 << 16, "SIMD"]]; - const CAP_INTERNAL = 1 << 10, CAP_SPIRAM = 1 << 11, CAP_DEFAULT = (1 << 3) | (1 << 2) | (1 << 1); // 8BIT | 32BIT | DMA-ish bits of MALLOC_CAP_DEFAULT + // (bit values from esp_heap_caps.h; the low bits EXEC / 32BIT / 8BIT / DMA + // qualify a region rather than name it and are left out of the label) + const HEAP_CAPS = [[1 << 10, "SPIRAM"], [1 << 11, "internal"], [1 << 12, "default"], [1 << 13, "IRAM 8-bit"], + [1 << 14, "retention"], [1 << 15, "RTC RAM"], [1 << 16, "SPM / TCM"], [1 << 17, "AHB DMA desc"], + [1 << 18, "AXI DMA desc"], [1 << 19, "cache-aligned"], [1 << 20, "SIMD"], + [1 << 21, "SPIRAM (no enc)"], [1 << 31, "invalid"]]; // =================================================================== // Elements & logging @@ -789,6 +797,7 @@

Log

consoleCarry = ""; moduleReady = false; // action buttons stay disabled until discovery resolved the ids monitorPresent = false; + resetDeviceState(); // nothing from a previous device may carry over updateUI(); setStatus("connected", "Connected"); els.devInfo.textContent = "Connected: " + t.name; @@ -803,6 +812,19 @@

Log

let capabilities = 0; let streaming = false; + // Forget everything learned from the previous device (capabilities gate + // the reboot buttons; info / heap / tasks are its data). + function resetDeviceState() { + capabilities = 0; + els.info.textContent = ""; + els.infoStatus.textContent = ""; + els.gauges.textContent = ""; + els.monitorStatus.textContent = ""; + lastTasks = []; + els.taskTable.querySelector("tbody").textContent = ""; + els.taskTable.hidden = true; + els.taskNote.hidden = true; + } function updateUI() { const connected = !!transport; els.usbBtn.textContent = connected && transport.kind === "usb" ? "Disconnect" : "Connect WebUSB"; @@ -1019,11 +1041,21 @@

Log

return label + " module: #" + r.id + " (" + how + ")"; } // Adopt the system + monitor module ids from a discovery reply (null = - // none) -- every frame built / matched from here on uses them. ?module=N - // applies to the system module only; the monitor module is optional and is - // located purely by discovery (absent -> its panel stays hidden). + // none) -- every frame built / matched from here on uses them. Both + // services advertise this page as their app, so the Device Hub links it + // with either module's id: ?module=N is applied to whichever of the two + // idents module N advertises the protocol of (the monitor's if N speaks + // espp.monitor, else the system's -- also when the device did not answer + // discovery, as in every other console). The monitor module is optional + // and counts as present only on a protocol match, never by the app / name + // fallback (the app is shared) -- absent, its panel stays hidden. function adoptModules(info) { - SYSTEM_IDENT.override = moduleOverrideFromQuery(location.search); + const override = moduleOverrideFromQuery(location.search); + const list = (info && Array.isArray(info.modules)) ? info.modules : []; + const overrideIsMonitor = override != null && + list.some((m) => m.id === override && m.protocol === MONITOR_PROTOCOL); + SYSTEM_IDENT.override = overrideIsMonitor ? null : override; + MONITOR_IDENT.override = overrideIsMonitor ? override : null; const r = resolveModuleId(info, SYSTEM_IDENT); moduleSystem = r.id; logLine("sys", describeModuleChoice("System", r, SYSTEM_IDENT, info)); @@ -1031,7 +1063,7 @@

Log

for (const n of r.notes) logLine("sys", n + "."); const m = resolveModuleId(info, MONITOR_IDENT); moduleMonitor = m.id; - monitorPresent = m.source !== "default"; + monitorPresent = m.source === "protocol" || (m.source === "override" && overrideIsMonitor); if (monitorPresent) logLine("sys", describeModuleChoice("Monitor", m, MONITOR_IDENT, info)); else logLine("sys", "Monitor module: not advertised by this device (heap / task panel hidden)."); moduleReady = true; @@ -1198,8 +1230,8 @@

Log

// Monitor: heap gauges + task table, on request or streamed // =================================================================== function heapName(flags) { - const names = HEAP_CAPS.filter(([bit]) => flags & bit).map(([, n]) => n); - if (!names.length) names.push((flags & CAP_DEFAULT) ? "default" : "caps 0x" + flags.toString(16)); + const names = HEAP_CAPS.filter(([bit]) => (flags & bit) !== 0).map(([, n]) => n); + if (!names.length) names.push("caps 0x" + (flags >>> 0).toString(16)); return names.join(" + "); } function parseHeap(payload) { @@ -1313,10 +1345,17 @@

Log

els.taskNote.textContent = lastTasks.length ? "" : "No tasks reported: the firmware needs CONFIG_FREERTOS_USE_TRACE_FACILITY and CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS."; } + function updateSortHeaders() { + for (const th of els.taskTable.querySelectorAll("th[data-key]")) { + if (th.dataset.key === taskSort.key) th.setAttribute("aria-sort", taskSort.dir > 0 ? "ascending" : "descending"); + else th.removeAttribute("aria-sort"); + } + } for (const th of els.taskTable.querySelectorAll("th[data-key]")) { - th.addEventListener("click", () => { + th.querySelector("button.sort").addEventListener("click", () => { const key = th.dataset.key; taskSort = { key, dir: taskSort.key === key ? -taskSort.dir : (key === "name" ? 1 : -1) }; + updateSortHeaders(); renderTaskTable(); }); } diff --git a/doc/Doxyfile b/doc/Doxyfile index 5924a853bc..f60edb835a 100755 --- a/doc/Doxyfile +++ b/doc/Doxyfile @@ -185,10 +185,10 @@ EXAMPLE_PATH = \ $(PROJECT_PATH)/components/st25dv/example/main/st25dv_example.cpp \ $(PROJECT_PATH)/components/st7123touch/example/main/st7123touch_example.cpp \ $(PROJECT_PATH)/components/state_machine/example/main/hfsm_example.cpp \ - $(PROJECT_PATH)/components/system/example/main/system_example.cpp \ $(PROJECT_PATH)/components/stream_frame/example/main/stream_frame_example.cpp \ $(PROJECT_PATH)/components/switch_pro/example/main/switch_pro_example.cpp \ $(PROJECT_PATH)/components/sx126x/example/main/sx126x_example.cpp \ + $(PROJECT_PATH)/components/system/example/main/system_example.cpp \ $(PROJECT_PATH)/components/tabulate/example/main/tabulate_example.cpp \ $(PROJECT_PATH)/components/t-deck/example/main/t_deck_example.cpp \ $(PROJECT_PATH)/components/t-dongle-s3/example/main/t_dongle_s3_example.cpp \ @@ -467,15 +467,15 @@ INPUT = \ $(PROJECT_PATH)/components/stream_frame/include/stream_frame.hpp \ $(PROJECT_PATH)/components/switch_pro/include/switch_pro.hpp \ $(PROJECT_PATH)/components/sx126x/include/sx126x.hpp \ + $(PROJECT_PATH)/components/system/include/system_control.hpp \ + $(PROJECT_PATH)/components/system/include/system_info.hpp \ + $(PROJECT_PATH)/components/system/include/system_service.hpp \ $(PROJECT_PATH)/components/t-deck/include/t-deck.hpp \ $(PROJECT_PATH)/components/t-dongle-s3/include/t-dongle-s3.hpp \ $(PROJECT_PATH)/components/t_keyboard/include/t_keyboard.hpp \ $(PROJECT_PATH)/components/tabulate/include/tabulate.hpp \ $(PROJECT_PATH)/components/task/include/task.hpp \ $(PROJECT_PATH)/components/task/include/run_on_core.hpp \ - $(PROJECT_PATH)/components/system/include/system_control.hpp \ - $(PROJECT_PATH)/components/system/include/system_info.hpp \ - $(PROJECT_PATH)/components/system/include/system_service.hpp \ $(PROJECT_PATH)/components/telemetry/include/telemetry.hpp \ $(PROJECT_PATH)/components/thermistor/include/thermistor.hpp \ $(PROJECT_PATH)/components/thread_pool/include/qos_band.hpp \ diff --git a/doc/en/core/monitor.rst b/doc/en/core/monitor.rst index aaed1af913..5fe3e02632 100644 --- a/doc/en/core/monitor.rst +++ b/doc/en/core/monitor.rst @@ -77,7 +77,9 @@ an instance and hosts find it through discovery by its protocol id). have are left out), ``GET_TASKS`` with the ``TaskMonitor`` table (name, CPU %, stack high-water mark, priority, core — it needs ``CONFIG_FREERTOS_USE_TRACE_FACILITY`` and -``CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS``, else the list is empty), and +``CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS``, else the list is empty; the reply +is capped so the whole frame fits ``Config::max_frame_bytes``, 4096 by +default, tasks beyond it being dropped from the end), and ``SET_STREAM`` starts a task that sends either or both periodically so a host can plot them live. The wire codec (``detail/monitor_protocol.hpp``) is host-buildable and unit-tested (``test/monitor_host_test.cpp``). The hosted diff --git a/doc/en/system/system_example.md b/doc/en/system/system_example.md index 2842ab6627..4ae87dfbf8 100644 --- a/doc/en/system/system_example.md +++ b/doc/en/system/system_example.md @@ -1,42 +1,2 @@ -# System Example - -[![Badge](https://components.espressif.com/components/espp/system/badge.svg)](https://components.espressif.com/components/espp/system) - -This example shows how to use the `espp::SystemService` and -`espp::MonitorService` components to expose device info / control and live -heap + task statistics over the native USB port (WebUSB + Web Serial) of an -ESP32-S3, for the hosted system console web app. - -## How to use example - -### Hardware Required - -An ESP32-S3 board with its native USB port connected to the host. - -### Build and Flash - -Build the project and flash it to the board, then run monitor tool to view serial output: - -``` -idf.py -p PORT flash monitor -``` - -(Replace PORT with the name of the serial port to use.) - -(To exit the serial monitor, type ``Ctrl-]``.) - -See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects. - -## Example Output - -``` -I (317) System Example: Starting system info + control example -I (327) System Example: System: -ESP32-S3 rev 0.2 (2 cores), ESP-IDF v6.1 -app: system_example 1 built Sep 30 2026 12:34:56 -partition: running 'factory', boot 'factory', OTA state n/a -reset: power-on; uptime 320 ms; MAC 34:85:18:xx:xx:xx -flash 8192 KiB, PSRAM 0 KiB, CPU 240 MHz, heap free 318412 (min 318412) -I (357) System Example: Reboot into the bootloader is supported on this chip -I (1077) System Example: Ready. Connect the native USB port and open the system console ... +```{include} ../../../components/system/example/README.md ``` From f80a2c854a1aea153d1bbcf16d4ccc60c25111f2 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 11:25:34 -0400 Subject: [PATCH 4/7] fix(system,monitor): review round 2 (query mask on the wire, errno codes, strict delay payload, dialog focus) - MonitorService: HEAP records carry the MALLOC_CAP_* mask the region was queried with, not the heap's own flags. - Both services: ERROR codes are the POSIX errno of the chosen std::errc (std::make_error_code(errc).value()); documented in both protocol headers and the README. - system_protocol: decode_delay() accepts an empty payload or exactly [delay_ms u16]; any other size is malformed (test added). - Console: the confirm dialog is aria-modal, Escape cancels, Tab cycles its two buttons, and focus returns to the button that opened it on close. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- .../include/detail/monitor_protocol.hpp | 2 ++ .../monitor/include/monitor_service.hpp | 8 ++++++-- components/system/README.md | 2 +- .../system/include/detail/system_protocol.hpp | 9 ++++++--- components/system/include/system_service.hpp | 8 +++++--- components/system/test/system_host_test.cpp | 2 ++ components/system/web/system_console.html | 20 +++++++++++++++++-- 7 files changed, 40 insertions(+), 11 deletions(-) diff --git a/components/monitor/include/detail/monitor_protocol.hpp b/components/monitor/include/detail/monitor_protocol.hpp index 4c34f26f95..8deb8e8791 100644 --- a/components/monitor/include/detail/monitor_protocol.hpp +++ b/components/monitor/include/detail/monitor_protocol.hpp @@ -22,6 +22,8 @@ // [priority u8][core i8]} // 0x83 OK [request_type u8] // 0x84 ERROR [request_type u8][code u32][utf8 message] +// code = the POSIX errno value of the std::errc the service +// chose (informational; the message is authoritative) // HEAP / TASKS answer the matching GET_* request and are also sent // unsolicited while streaming is enabled (same encoding, so a host decodes // both the same way). A TASKS payload is capped (MonitorService: to fit diff --git a/components/monitor/include/monitor_service.hpp b/components/monitor/include/monitor_service.hpp index aed6bfb82b..9c5db922ce 100644 --- a/components/monitor/include/monitor_service.hpp +++ b/components/monitor/include/monitor_service.hpp @@ -210,7 +210,9 @@ class MonitorService : public BaseComponent { const HeapMonitor::HeapInfo hi = HeapMonitor::get_info(flags); if (hi.total_size == 0) continue; // e.g. MALLOC_CAP_SPIRAM on a chip without PSRAM - regions.push_back({.flags = static_cast(hi.heap_flags), + // the wire carries the QUERY mask (what the region was asked for), so a + // host can label it the way it configured the service + regions.push_back({.flags = static_cast(flags), .free_bytes = static_cast(hi.free_bytes), .min_free_bytes = static_cast(hi.min_free_bytes), .largest_free_block = static_cast(hi.largest_free_block), @@ -307,7 +309,9 @@ class MonitorService : public BaseComponent { namespace proto = espp::detail::monitor_protocol; logger_.warn("{} (type 0x{:02x})", message, request_type); send_frame(proto::build_frame( - Type::Error, proto::encode_error(request_type, static_cast(errc), message), + Type::Error, + proto::encode_error(request_type, static_cast(std::make_error_code(errc).value()), + message), module_id())); } diff --git a/components/system/README.md b/components/system/README.md index 5c02f16323..ab99e116fb 100644 --- a/components/system/README.md +++ b/components/system/README.md @@ -54,7 +54,7 @@ little-endian. See `include/detail/system_protocol.hpp` (host-buildable, with | `0x03` REBOOT_TO_BOOTLOADER | H→D | `[delay_ms u16]` — reply OK, restart into download mode | | `0x81` INFO | D→H | tagged records `[tag u8][len u8][value]` (unknown tags are skipped) | | `0x83` OK | D→H | `[request_type u8]` | -| `0x84` ERROR | D→H | `[request_type u8][code u32][utf8 message]` | +| `0x84` ERROR | D→H | `[request_type u8][code u32][utf8 message]` — code is the POSIX errno of the chosen `std::errc` (the message is authoritative) | INFO tags: 1 chip model (str), 2 chip revision (u16), 3 cores (u8), 4 chip features (u32), 5 IDF version, 6 project name, 7 app version, 8 build date, 9 diff --git a/components/system/include/detail/system_protocol.hpp b/components/system/include/detail/system_protocol.hpp index 81add2c19e..23108b63bb 100644 --- a/components/system/include/detail/system_protocol.hpp +++ b/components/system/include/detail/system_protocol.hpp @@ -19,6 +19,8 @@ // version bump (see InfoTag for the values). // 0x83 OK [request_type u8] // 0x84 ERROR [request_type u8][code u32][utf8 message] +// code = the POSIX errno value of the std::errc the service +// chose (informational; the message is authoritative) // A reboot request is acknowledged with OK first; the device restarts after // the requested delay (clamped to at least Config::min_restart_delay). @@ -309,12 +311,13 @@ inline std::vector encode_delay(uint16_t delay_ms) { return p; } -/// Decode a REBOOT / REBOOT_TO_BOOTLOADER payload (an empty payload means 0 ms). +/// Decode a REBOOT / REBOOT_TO_BOOTLOADER payload: empty = 0 ms, else exactly +/// [delay_ms u16]; any other size is malformed (nullopt). inline std::optional decode_delay(std::span p) { if (p.empty()) return 0; - if (p.size() < 2) - return std::nullopt; + if (p.size() != 2) + return std::nullopt; // exactly [delay_ms u16]; anything else is malformed return espp::stream_frame::get_u16(p); } diff --git a/components/system/include/system_service.hpp b/components/system/include/system_service.hpp index 7c3da30722..f2abda4f46 100644 --- a/components/system/include/system_service.hpp +++ b/components/system/include/system_service.hpp @@ -266,9 +266,11 @@ class SystemService : public BaseComponent { void send_error(uint8_t request_type, std::errc errc, std::string_view message) { namespace proto = espp::detail::system_protocol; logger_.warn("{} (type 0x{:02x})", message, request_type); - send(proto::build_frame(Type::Error, - proto::encode_error(request_type, static_cast(errc), message), - module_id())); + send(proto::build_frame( + Type::Error, + proto::encode_error(request_type, static_cast(std::make_error_code(errc).value()), + message), + module_id())); } private: diff --git a/components/system/test/system_host_test.cpp b/components/system/test/system_host_test.cpp index 017ebd4e03..13db81cbc7 100644 --- a/components/system/test/system_host_test.cpp +++ b/components/system/test/system_host_test.cpp @@ -137,6 +137,8 @@ static void test_requests_and_replies() { CHECK(sp::decode_delay({}) == 0); // empty payload = no delay const uint8_t one[] = {1}; CHECK(!sp::decode_delay(one)); + const uint8_t three[] = {1, 2, 3}; // too long is malformed too, not "the first two bytes" + CHECK(!sp::decode_delay(three)); CHECK(sp::encode_ok(0x02) == std::vector{0x02}); const auto e = sp::encode_error(0x03, 95, "not supported"); CHECK(e.size() == 5 + 13 && e[0] == 3 && e[1] == 95 && e[2] == 0 && e[5] == 'n'); diff --git a/components/system/web/system_console.html b/components/system/web/system_console.html index c12e61ab08..8d40de3db0 100644 --- a/components/system/web/system_console.html +++ b/components/system/web/system_console.html @@ -247,7 +247,7 @@

Control

-
+
@@ -1189,14 +1189,30 @@

Log

// Reboot control, with an in-page confirmation step // =================================================================== let confirmAction = null; + let confirmOpener = null; // the button that opened the dialog: focus returns to it function openConfirm(text, action) { confirmAction = action; + confirmOpener = document.activeElement instanceof HTMLElement ? document.activeElement : null; els.confirmText.textContent = text; els.confirm.classList.add("open"); els.confirmYes.focus(); } - function closeConfirm() { confirmAction = null; els.confirm.classList.remove("open"); } + function closeConfirm() { + const wasOpen = els.confirm.classList.contains("open"); + confirmAction = null; + els.confirm.classList.remove("open"); + if (wasOpen && confirmOpener && confirmOpener.isConnected && !confirmOpener.disabled) confirmOpener.focus(); + confirmOpener = null; + } els.confirmNo.addEventListener("click", closeConfirm); + // Escape cancels; Tab stays inside the two buttons while the dialog is open + els.confirm.addEventListener("keydown", (e) => { + if (e.key === "Escape") { e.preventDefault(); closeConfirm(); return; } + if (e.key === "Tab") { + e.preventDefault(); + (document.activeElement === els.confirmYes ? els.confirmNo : els.confirmYes).focus(); + } + }); els.confirmYes.addEventListener("click", async () => { const action = confirmAction; closeConfirm(); From fc601c55850be0a864caff6a83bee558d2cf8b16 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 11:38:16 -0400 Subject: [PATCH 5/7] fix(system): USB persistence across the bootloader reset is opt-in (off by default) The S2 / S3 ROM only expects a persisted USB peripheral from an application whose USB device is ROM-CDC/DFU-compatible (ESP-IDF's ROM USB console); persisting a TinyUSB vendor + CDC composite can leave the host with a stale enumeration the bootloader cannot serve. reboot_to_bootloader() therefore no longer sets USBDC_PERSIST_ENA unconditionally: SystemBootloaderOptions (SystemControl::BootloaderOptions) / SystemService::Config::usb_persist opt in, default false, and when enabled the ROM prep runs the way usb_console.c does (usb_dc_prepare_persist() then chip_usb_set_persist_flags()), S2 / S3 only. Header, README and system.rst document when persistence applies. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/system/README.md | 13 +++- components/system/include/system_control.hpp | 68 +++++++++++++++----- components/system/include/system_service.hpp | 8 ++- doc/en/system/system.rst | 25 +++++-- 4 files changed, 89 insertions(+), 25 deletions(-) diff --git a/components/system/README.md b/components/system/README.md index ab99e116fb..e48678d5d8 100644 --- a/components/system/README.md +++ b/components/system/README.md @@ -15,9 +15,16 @@ I restart it?" for any espp application: sets the chip's *force download boot* flag and restarts, so the device comes back in the ROM bootloader's download mode ready for `esptool` / `idf.py flash` (what holding the BOOT strap does, without a button). Supported on the - ESP32-S2 / -S3 (the ROM's USB CDC / DFU device stays attached), -C2 / -C3 / - -C5 / -C6 / -C61 / -H2 / -H21 and -P4 (USB-Serial-JTAG); the classic ESP32 - has no software path and reports `operation_not_supported`. + ESP32-S2 / -S3 (the ROM enumerates its USB CDC / DFU device afresh after the + reset), -C2 / -C3 / -C5 / -C6 / -C61 / -H2 / -H21 and -P4 (USB-Serial-JTAG); + the classic ESP32 has no software path and reports `operation_not_supported`. + On the S2 / S3 the ROM can also keep the USB peripheral's state across the + reset (`BootloaderOptions::usb_persist`, `SystemService::Config::usb_persist`) + so the host sees no re-plug — **opt-in, off by default**: the ROM only + expects that from an application whose USB device is ROM-CDC/DFU-compatible + (ESP-IDF's ROM USB console); a TinyUSB vendor + CDC composite like the espp + examples has different descriptors, and persisting it can leave the host + with a stale enumeration the bootloader cannot serve. - `espp::SystemService` — both of the above as a transport-agnostic [dispatcher](../dispatcher) module (`espp.system` v1, module 7 by default): `GET_INFO` answers with a tagged-record snapshot hosts can extend-proof diff --git a/components/system/include/system_control.hpp b/components/system/include/system_control.hpp index d251830a40..20698f2a6d 100644 --- a/components/system/include/system_control.hpp +++ b/components/system/include/system_control.hpp @@ -24,18 +24,33 @@ #include "soc/lp_system_reg.h" #define ESPP_SYSTEM_DOWNLOAD_BOOT_LP_SYSTEM 1 #endif +// The ROM USB persistence calls (usb_dc_prepare_persist() + +// chip_usb_set_persist_flags()) exist on the S2 / S3 only, whose ROM has a USB +// CDC / DFU device of its own; they are used only when a caller opts in. #if CONFIG_IDF_TARGET_ESP32S2 #include "esp32s2/rom/usb/chip_usb_dw_wrapper.h" +#include "esp32s2/rom/usb/usb_dc.h" #include "esp32s2/rom/usb/usb_persist.h" #define ESPP_SYSTEM_USB_PERSIST 1 #elif CONFIG_IDF_TARGET_ESP32S3 #include "esp32s3/rom/usb/chip_usb_dw_wrapper.h" +#include "esp32s3/rom/usb/usb_dc.h" #include "esp32s3/rom/usb/usb_persist.h" #define ESPP_SYSTEM_USB_PERSIST 1 #endif namespace espp { +/// Options for SystemControl::reboot_to_bootloader() (a namespace-scope type +/// so it is complete where the functions default it). +struct SystemBootloaderOptions { + /// Keep the USB peripheral's state across the reset (ESP32-S2 / -S3 ROM + /// only; ignored elsewhere). ONLY for applications whose USB device is + /// ROM-CDC/DFU-compatible (see the SystemControl notes); off by default, + /// letting the ROM re-enumerate on its own. + bool usb_persist{false}; +}; + /** * @brief Restart control: a plain reboot, and a reboot into the ROM * bootloader's download (serial flashing) mode. @@ -45,12 +60,22 @@ namespace espp { * download mode instead of running the app -- exactly what holding the BOOT * strap during a reset does, without touching a button. The device then * re-enumerates as the ROM's own flashing interface: the USB CDC / DFU device - * on the ESP32-S2 / -S3 native USB port (the ROM's USB stack is kept - * persistent across the reset), or USB-Serial-JTAG on the ESP32-C3 / -C6 / - * -H2 / -C5 / -C61 / -H21 / -P4 -- so `esptool` / `idf.py flash` can program - * it. On the classic ESP32 there is no software path (only the GPIO0 strap): - * bootloader_reboot_supported() is false and reboot_to_bootloader() fails - * with operation_not_supported. + * on the ESP32-S2 / -S3 native USB port, or USB-Serial-JTAG on the ESP32-C3 / + * -C6 / -H2 / -C5 / -C61 / -H21 / -P4 -- so `esptool` / `idf.py flash` can + * program it. On the classic ESP32 there is no software path (only the GPIO0 + * strap): bootloader_reboot_supported() is false and reboot_to_bootloader() + * fails with operation_not_supported. + * + * **USB persistence (S2 / S3, opt-in)**: by default the reset tears the USB + * connection down and the ROM enumerates its CDC / DFU device afresh, which + * works for any application. The ROM can instead keep the USB peripheral's + * state across the reset (BootloaderOptions::usb_persist), so the host sees + * no re-plug -- but the ROM only expects that from an application whose USB + * device is ROM-CDC/DFU-compatible (the same descriptors the ROM exposes, as + * ESP-IDF's ROM USB console has); a TinyUSB composite device (the espp + * examples: vendor + CDC) has different descriptors, and persisting it can + * leave the host with a stale enumeration the bootloader cannot serve. Leave + * it off unless the application runs on the ROM USB console driver. * * Both functions restart immediately; use the delayed variants (or * espp::SystemService, which replies before restarting) when a reply must @@ -58,6 +83,9 @@ namespace espp { */ class SystemControl { public: + /// Options for the reboot into download mode (see SystemBootloaderOptions). + using BootloaderOptions = SystemBootloaderOptions; + /// @brief Whether reboot_to_bootloader() is implemented for this chip. static constexpr bool bootloader_reboot_supported() { #if ESPP_SYSTEM_DOWNLOAD_BOOT_RTC_CNTL || ESPP_SYSTEM_DOWNLOAD_BOOT_LP_AON || \ @@ -74,14 +102,15 @@ class SystemControl { /// @brief Restart into the ROM bootloader's download mode. /// @param ec Set to operation_not_supported on chips without a software path /// (classic ESP32); then returns false without restarting. + /// @param options See BootloaderOptions (USB persistence is opt-in). /// @return Does not return on success; false on failure. - static bool reboot_to_bootloader(std::error_code &ec) { + static bool reboot_to_bootloader(std::error_code &ec, const BootloaderOptions &options = {}) { ec.clear(); if (!bootloader_reboot_supported()) { ec = std::make_error_code(std::errc::operation_not_supported); return false; } - arm_download_boot(); + arm_download_boot(options); esp_restart(); return true; // not reached } @@ -97,15 +126,16 @@ class SystemControl { /// @brief Restart into download mode after a delay, from a detached thread. /// Returns immediately; false (nothing scheduled) if unsupported. - static bool reboot_to_bootloader_after(std::chrono::milliseconds delay, std::error_code &ec) { + static bool reboot_to_bootloader_after(std::chrono::milliseconds delay, std::error_code &ec, + const BootloaderOptions &options = {}) { ec.clear(); if (!bootloader_reboot_supported()) { ec = std::make_error_code(std::errc::operation_not_supported); return false; } - std::thread([delay]() { + std::thread([delay, options]() { std::this_thread::sleep_for(delay); - arm_download_boot(); + arm_download_boot(options); esp_restart(); }).detach(); return true; @@ -113,11 +143,19 @@ class SystemControl { private: /// Set the chip's force-download-boot flag (survives the reset that follows). - static void arm_download_boot() { + static void arm_download_boot(const BootloaderOptions &options) { #if ESPP_SYSTEM_USB_PERSIST - // keep the ROM's USB device attached across the reset so the host sees the - // download-mode CDC / DFU interface without a full re-plug - chip_usb_set_persist_flags(USBDC_PERSIST_ENA); + if (options.usb_persist) { + // Keep the ROM USB device attached across the reset, the way ESP-IDF's + // ROM USB console does before its own reboot-to-bootloader: park the + // peripheral (usb_dc_prepare_persist(), "reboot soon after") and set the + // persist flag the ROM reads on the next boot. Opt-in only: see the + // class notes. + usb_dc_prepare_persist(); + chip_usb_set_persist_flags(USBDC_PERSIST_ENA); + } +#else + (void)options; #endif #if ESPP_SYSTEM_DOWNLOAD_BOOT_RTC_CNTL REG_WRITE(RTC_CNTL_OPTION1_REG, RTC_CNTL_FORCE_DOWNLOAD_BOOT); diff --git a/components/system/include/system_service.hpp b/components/system/include/system_service.hpp index f2abda4f46..eec956fc74 100644 --- a/components/system/include/system_service.hpp +++ b/components/system/include/system_service.hpp @@ -92,6 +92,12 @@ class SystemService : public BaseComponent { uint8_t module{kModule}; bool allow_reboot{true}; ///< Serve REBOOT (else ERROR "not permitted"). bool allow_bootloader{true}; ///< Serve REBOOT_TO_BOOTLOADER (else ERROR "not permitted"). + /// Keep the USB peripheral's state across a REBOOT_TO_BOOTLOADER reset + /// (ESP32-S2 / -S3 ROM only; see SystemControl::BootloaderOptions). Off by + /// default: the ROM re-enumerates its CDC / DFU device on its own, which is + /// right for a TinyUSB (vendor / CDC composite) application; enable only + /// when the application's USB device is ROM-CDC/DFU-compatible. + bool usb_persist{false}; /// Optional veto for a specific reboot request; called after the allow_* /// checks, outside the lock. nullptr = every permitted request proceeds. reboot_request_fn on_reboot_request{nullptr}; @@ -240,7 +246,7 @@ class SystemService : public BaseComponent { logger_.info("{} in {} ms", bootloader ? "rebooting into the bootloader" : "rebooting", wait.count()); if (bootloader) - SystemControl::reboot_to_bootloader_after(wait, ec); + SystemControl::reboot_to_bootloader_after(wait, ec, {.usb_persist = config_.usb_persist}); else SystemControl::reboot_after(wait); return true; diff --git a/doc/en/system/system.rst b/doc/en/system/system.rst index a05223a6b2..5dc843b045 100644 --- a/doc/en/system/system.rst +++ b/doc/en/system/system.rst @@ -16,12 +16,25 @@ flag in its always-on register and restarts, so the next boot stays in the ROM download mode instead of running the app — what holding the BOOT strap during a reset does, without a button. The device then re-enumerates as the ROM's own flashing interface (the USB CDC / DFU device on the ESP32-S2 / -S3 -native USB port, kept attached across the reset; USB-Serial-JTAG on the -ESP32-C3 / -C6 / -H2 / -C5 / -C61 / -H21 / -P4), ready for ``esptool`` / -``idf.py flash``. The classic ESP32 has no software path (only the GPIO0 -strap): ``bootloader_reboot_supported()`` is false there and the call fails -with ``operation_not_supported``. The ``*_after(delay)`` variants restart from -a detached thread so a reply can leave the transport first. +native USB port; USB-Serial-JTAG on the ESP32-C3 / -C6 / -H2 / -C5 / -C61 / +-H21 / -P4), ready for ``esptool`` / ``idf.py flash``. The classic ESP32 has +no software path (only the GPIO0 strap): ``bootloader_reboot_supported()`` is +false there and the call fails with ``operation_not_supported``. The +``*_after(delay)`` variants restart from a detached thread so a reply can +leave the transport first. + +By default the reset tears the USB connection down and the ROM enumerates its +device afresh, which works for any application. On the ESP32-S2 / -S3 the ROM +can instead keep the USB peripheral's state across the reset +(``SystemControl::BootloaderOptions::usb_persist``, passed through as +``SystemService::Config::usb_persist``) so the host sees no re-plug. This is +**opt-in and off by default**: the ROM only expects it from an application +whose USB device is ROM-CDC/DFU-compatible (ESP-IDF's ROM USB console driver, +which is what performs the same sequence — ``usb_dc_prepare_persist()`` then +the persist flag — before its own reboot into the bootloader). A TinyUSB +composite device such as the espp examples' vendor + CDC has different +descriptors, and persisting it can leave the host with a stale enumeration +the bootloader cannot serve; leave the option off there. The `SystemService` class serves both over **any byte stream** as a :doc:`dispatcher <../dispatcher/dispatcher>` module (``espp.system`` v1, From 47cd98573305966e666dee685f3d7b416196a2c3 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 16:28:43 -0400 Subject: [PATCH 6/7] fix(system,monitor): previously-missed review findings (frame cap on HEAP, stream wait predicate, correlation ids, serial teardown) - MonitorService: one max_payload() (max_frame_bytes less the largest header + CRC) caps HEAP as well as TASKS; encode_heap() takes the cap and reports what fit, the overflow is logged rate-limited. Docs + host test. - MonitorService: the stream task uses the (mutex, cv, notified) callback, waits on the notified predicate and clears it under the mutex, so a spurious wake-up no longer emits events early. - Both services echo the request frame's correlation id on every reply (INFO / HEAP / TASKS / OK / ERROR); streamed events carry none. The console stamps each request with a fresh u16 id and pairs replies by it: a mismatching correlated reply is dropped as stale, an uncorrelated reply is accepted only until the device has shown it echoes ids (older firmware). Protocol headers, READMEs and rst pages say so; host tests cover the echo through the codec. - Console Web Serial: a failing getWriter() closes the port it just opened before rethrowing; close() finishes / aborts the writable stream before releasing its lock so the port cannot stay locked and open. - Console INFO decoder rejects a payload ending in a lone tag byte (incomplete record header), like the C++ decoder. - System example builds with C++20 like the other examples. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/monitor/README.md | 6 +- .../include/detail/monitor_protocol.hpp | 38 +++++++--- .../monitor/include/monitor_service.hpp | 71 ++++++++++++------- components/monitor/test/monitor_host_test.cpp | 25 +++++++ components/system/README.md | 4 ++ components/system/example/CMakeLists.txt | 2 +- .../system/include/detail/system_protocol.hpp | 9 ++- components/system/include/system_service.hpp | 28 +++++--- components/system/test/system_host_test.cpp | 16 +++++ components/system/web/system_console.html | 61 +++++++++++++--- doc/en/core/monitor.rst | 10 +-- doc/en/system/system.rst | 4 +- 12 files changed, 205 insertions(+), 69 deletions(-) diff --git a/components/monitor/README.md b/components/monitor/README.md index b4e2a94009..0a84ce93d4 100644 --- a/components/monitor/README.md +++ b/components/monitor/README.md @@ -32,8 +32,10 @@ statistics over any framed byte stream as an `espp::Dispatcher` module (`espp.monitor` v1, module 8 by default): `GET_HEAP` (one record per configured `MALLOC_CAP_*` region), `GET_TASKS` (the `TaskMonitor` table; needs `CONFIG_FREERTOS_USE_TRACE_FACILITY` + `CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS`; -capped so the whole frame fits `Config::max_frame_bytes`, 4096 by default) -and `SET_STREAM` (periodic HEAP / TASKS events). The wire codec lives in +capped, like `GET_HEAP`, so the whole frame fits `Config::max_frame_bytes`, 4096 by default) +and `SET_STREAM` (periodic HEAP / TASKS events). Replies echo the request +frame's correlation id, so a host can pair them and drop stale ones; streamed +events carry none. The wire codec lives in `include/detail/monitor_protocol.hpp` (host-buildable, tested by `test/monitor_host_test.cpp`). The hosted [system console](https://esp-cpp.github.io/espp/apps/system_console.html) web diff --git a/components/monitor/include/detail/monitor_protocol.hpp b/components/monitor/include/detail/monitor_protocol.hpp index 8deb8e8791..6efadc7669 100644 --- a/components/monitor/include/detail/monitor_protocol.hpp +++ b/components/monitor/include/detail/monitor_protocol.hpp @@ -26,10 +26,13 @@ // chose (informational; the message is authoritative) // HEAP / TASKS answer the matching GET_* request and are also sent // unsolicited while streaming is enabled (same encoding, so a host decodes -// both the same way). A TASKS payload is capped (MonitorService: to fit -// Config::max_frame_bytes with the frame overhead; at most the payload limit): -// tasks that would not fit are dropped from the END of the list. +// both the same way). Replies echo the request frame's correlation id (if it +// carried one); streamed events carry none. HEAP and TASKS payloads are capped +// (MonitorService: to fit Config::max_frame_bytes with the frame overhead; at +// most the payload limit): +// regions / tasks that would not fit are dropped from the END of the list. +#include #include #include #include @@ -107,22 +110,35 @@ inline constexpr bool is_reply(Type type) { return (static_cast(type) & /// Build an encoded frame for a monitor message (device->host types map to the /// frame reply flag). +/// @param correlation The stream_frame correlation id to carry: a reply echoes +/// the request's (so a host can pair them), a streamed event carries none. inline std::vector build_frame(Type type, std::span payload = {}, - uint8_t module = kModule) { + uint8_t module = kModule, + std::optional correlation = std::nullopt) { return espp::stream_frame::build_frame(is_reply(type), module, static_cast(type), - payload); + payload, correlation); } // ---- encoders --------------------------------------------------------------- -/// Encode a HEAP payload. At most 255 regions are encoded. -inline std::vector encode_heap(std::span regions) { +/// Bytes one HeapRegion occupies on the wire. +inline constexpr size_t kHeapRegionSize = 24; + +/// Encode a HEAP payload, keeping it within @p max_bytes (the frame payload +/// limit by default): regions that would not fit are dropped from the end. +/// At most 255 regions are encoded. +/// @param[out] encoded_count Set to the number of regions encoded, if non-null. +inline std::vector encode_heap(std::span regions, + size_t max_bytes = espp::stream_frame::kMaxPayloadSize, + size_t *encoded_count = nullptr) { std::vector p; - const size_t n = regions.size() > 255 ? 255 : regions.size(); - p.reserve(1 + 24 * n); + const size_t fit = max_bytes > 1 ? (max_bytes - 1) / kHeapRegionSize : 0; + const size_t n = std::min({regions.size(), size_t{255}, fit}); + if (encoded_count) + *encoded_count = n; + p.reserve(1 + kHeapRegionSize * n); p.push_back(static_cast(n)); - for (size_t i = 0; i < n; ++i) { - const auto &r = regions[i]; + for (const auto &r : regions.first(n)) { espp::stream_frame::put_u32(p, r.flags); espp::stream_frame::put_u32(p, r.free_bytes); espp::stream_frame::put_u32(p, r.min_free_bytes); diff --git a/components/monitor/include/monitor_service.hpp b/components/monitor/include/monitor_service.hpp index 9c5db922ce..e2aca9651c 100644 --- a/components/monitor/include/monitor_service.hpp +++ b/components/monitor/include/monitor_service.hpp @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -88,8 +89,8 @@ class MonitorService : public BaseComponent { /// Shortest streaming period a host may request (SET_STREAM is clamped to it). std::chrono::milliseconds min_stream_period{100}; /// Largest encoded frame (header + payload + CRC) `send` can carry in one - /// write: the TASKS payload is capped so the whole frame fits (tasks that - /// do not fit are dropped from the end, logged). 4096 matches the default + /// write: the HEAP and TASKS payloads are capped so the whole frame fits + /// (regions / tasks that do not fit are dropped from the end, logged). 4096 matches the default /// TinyUSB vendor / CDC TX FIFO of the espp examples; the stream_frame /// maximum is kMaxFrameSize (4111). size_t max_frame_bytes{4096}; @@ -141,7 +142,7 @@ class MonitorService : public BaseComponent { void handle(const espp::stream_frame::Frame &frame) { if (frame.module != module_id() || frame.is_reply()) return; - handle_frame(frame.type, frame.payload); + handle_frame(frame.type, frame.payload, frame.correlation); } /// @brief Feed received transport bytes (standalone use, without a Dispatcher). @@ -166,24 +167,29 @@ class MonitorService : public BaseComponent { * @return true if the type belongs to the monitor protocol (a reply was * sent), false if it was ignored. */ - bool handle_frame(uint8_t type, std::span payload) { + /// @param correlation The request frame's correlation id, if it carried one; + /// every reply echoes it (streamed events carry none). + bool handle_frame(uint8_t type, std::span payload, + std::optional correlation = std::nullopt) { namespace proto = espp::detail::monitor_protocol; switch (static_cast(type)) { case Type::GetHeap: - send_frame(proto::build_frame(Type::Heap, build_heap(), module_id())); + send_frame(proto::build_frame(Type::Heap, build_heap(), module_id(), correlation)); return true; case Type::GetTasks: - send_frame(proto::build_frame(Type::Tasks, build_tasks(), module_id())); + send_frame(proto::build_frame(Type::Tasks, build_tasks(), module_id(), correlation)); return true; case Type::SetStream: { const auto req = proto::decode_set_stream(payload); if (!req) { send_error(type, std::errc::invalid_argument, - "malformed SET_STREAM (expected u8 enable, u16 period_ms, u8 what)"); + "malformed SET_STREAM (expected u8 enable, u16 period_ms, u8 what)", + correlation); return true; } if (req->enable && (req->what & (proto::kStreamHeap | proto::kStreamTasks)) == 0) { - send_error(type, std::errc::invalid_argument, "SET_STREAM: nothing selected to stream"); + send_error(type, std::errc::invalid_argument, "SET_STREAM: nothing selected to stream", + correlation); return true; } if (req->enable) @@ -192,7 +198,7 @@ class MonitorService : public BaseComponent { stop_stream(); logger_.debug("SET_STREAM enable={} period_ms={} what=0x{:02x}", req->enable, period_.load().count(), req->what); - send_frame(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id())); + send_frame(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id(), correlation)); return true; } default: @@ -201,8 +207,9 @@ class MonitorService : public BaseComponent { } protected: - /// The HEAP payload for the configured regions (regions with no memory left out). - std::vector build_heap() const { + /// The HEAP payload for the configured regions (regions with no memory left + /// out; capped so the frame fits Config::max_frame_bytes, the overflow logged). + std::vector build_heap() { namespace proto = espp::detail::monitor_protocol; std::vector regions; regions.reserve(config_.heap_regions.size()); @@ -219,7 +226,12 @@ class MonitorService : public BaseComponent { .allocated_bytes = static_cast(hi.allocated_bytes), .total_size = static_cast(hi.total_size)}); } - return proto::encode_heap(regions); + size_t encoded = 0; + auto payload = proto::encode_heap(regions, max_payload(), &encoded); + if (encoded < regions.size()) + logger_.warn_rate_limited("HEAP payload full: reporting {} of {} regions", encoded, + regions.size()); + return payload; } /// The TASKS payload (capped at the frame payload limit). @@ -239,7 +251,7 @@ class MonitorService : public BaseComponent { .core_id = static_cast(t.core_id)}; }); size_t encoded = 0; - auto payload = proto::encode_tasks(tasks, max_tasks_payload(), &encoded); + auto payload = proto::encode_tasks(tasks, max_payload(), &encoded); if (encoded < tasks.size()) logger_.warn_rate_limited("TASKS payload full: reporting {} of {} tasks", encoded, tasks.size()); @@ -249,15 +261,15 @@ class MonitorService : public BaseComponent { // report an empty list (and say why, once in a while) logger_.warn_rate_limited("task statistics need CONFIG_FREERTOS_USE_TRACE_FACILITY and " "CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS; reporting no tasks"); - return proto::encode_tasks({}, max_tasks_payload(), nullptr); + return proto::encode_tasks({}, max_payload(), nullptr); #endif } - /// The TASKS payload cap: Config::max_frame_bytes less the frame overhead - /// (a 9-byte header, no correlation id, plus the CRC), never above the codec's - /// own payload limit. - size_t max_tasks_payload() const { - constexpr size_t overhead = espp::stream_frame::kHeaderSize + espp::stream_frame::kCrcSize; + /// The payload cap shared by HEAP and TASKS: Config::max_frame_bytes less the + /// frame overhead (the largest header, i.e. with a correlation id echoed, + /// plus the CRC), never above the codec's own payload limit. + size_t max_payload() const { + constexpr size_t overhead = espp::stream_frame::kMaxHeaderSize + espp::stream_frame::kCrcSize; const size_t cap = config_.max_frame_bytes > overhead ? config_.max_frame_bytes - overhead : 0; return std::min(cap, espp::stream_frame::kMaxPayloadSize); } @@ -270,14 +282,17 @@ class MonitorService : public BaseComponent { if (task_) return; // already running: the new period / selection apply on its next wake task_ = std::make_unique( - Task::Config{.callback = [this](std::mutex &m, - std::condition_variable &cv) { return stream_step(m, cv); }, + Task::Config{.callback = [this](std::mutex &m, std::condition_variable &cv, + bool ¬ified) { return stream_step(m, cv, notified); }, .task_config = config_.task_config}); task_->start(); } - /// One streaming period: send the selected reports, then wait (interruptibly). - bool stream_step(std::mutex &m, std::condition_variable &cv) { + /// One streaming period: send the selected reports (no correlation id: they + /// are events, not replies), then wait one period. The wait uses the Task's + /// notified flag as its predicate, so a spurious wake-up does not emit early; + /// Task::stop() notifies, which ends the wait and the task. + bool stream_step(std::mutex &m, std::condition_variable &cv, bool ¬ified) { namespace proto = espp::detail::monitor_protocol; if (streaming_.load()) { const uint8_t what = what_.load(); @@ -287,8 +302,9 @@ class MonitorService : public BaseComponent { send_frame(proto::build_frame(Type::Tasks, build_tasks(), module_id())); } std::unique_lock lock(m); - cv.wait_for(lock, period_.load()); - return false; // keep running until stopped + cv.wait_for(lock, period_.load(), [¬ified] { return notified; }); + notified = false; // consumed, under the mutex, per the Task contract + return false; // keep running until stopped } /// Transmit a frame. Serialized on send_mutex_ (held across the callback) so @@ -305,14 +321,15 @@ class MonitorService : public BaseComponent { config_.send(frame); } - void send_error(uint8_t request_type, std::errc errc, std::string_view message) { + void send_error(uint8_t request_type, std::errc errc, std::string_view message, + std::optional correlation = std::nullopt) { namespace proto = espp::detail::monitor_protocol; logger_.warn("{} (type 0x{:02x})", message, request_type); send_frame(proto::build_frame( Type::Error, proto::encode_error(request_type, static_cast(std::make_error_code(errc).value()), message), - module_id())); + module_id(), correlation)); } private: diff --git a/components/monitor/test/monitor_host_test.cpp b/components/monitor/test/monitor_host_test.cpp index 09743c855f..84ff8893c7 100644 --- a/components/monitor/test/monitor_host_test.cpp +++ b/components/monitor/test/monitor_host_test.cpp @@ -57,6 +57,14 @@ static void test_heap_roundtrip() { // truncated: declared 2 regions, only one present CHECK(!mp::decode_heap(std::span(p.data(), 1 + 24))); CHECK(!mp::decode_heap({})); + // the cap drops whole regions from the end (the service passes its + // max_payload(); 1 count byte + 24 per region) + size_t nh = 0; + const auto capped = mp::encode_heap(regions, 1 + 24 + 5, &nh); + CHECK(nh == 1 && capped[0] == 1 && capped.size() == 25); + CHECK(mp::decode_heap(capped) && mp::decode_heap(capped)->size() == 1); + size_t nz = 9; + CHECK(mp::encode_heap(regions, 10, &nz).size() == 1 && nz == 0); // empty list const auto e = mp::encode_heap({}); CHECK(e.size() == 1 && e[0] == 0); @@ -148,6 +156,23 @@ static void test_frames() { const auto frames = parser.feed(stream); CHECK(frames.size() == 2 && !frames[0].is_reply() && frames[1].is_reply() && frames[1].module == 9 && frames[1].payload.size() == 1); + // correlation: a request may carry a u16 id; a reply built with the + // request's id (what MonitorService::handle_frame does for every reply) + // echoes it, a streamed event (built without one) carries none + const auto creq = mp::build_frame(mp::Type::GetTasks, {}, 8, 0xBEEF); + sf::StreamParser p2; + const auto cf = p2.feed(creq); + CHECK(cf.size() == 1 && cf[0].has_correlation() && *cf[0].correlation == 0xBEEF && + creq.size() == 9 + 2 + 4 && (creq[2] & 0x02) != 0); + const auto crep = mp::build_frame(mp::Type::Tasks, mp::encode_tasks({}), 8, cf[0].correlation); + const auto cr = sf::StreamParser{}.feed(crep); + CHECK(cr.size() == 1 && cr[0].is_reply() && cr[0].correlation == std::optional(0xBEEF)); + const auto ev = mp::build_frame(mp::Type::Heap, mp::encode_heap({}), 8); + const auto er = sf::StreamParser{}.feed(ev); + CHECK(er.size() == 1 && !er[0].has_correlation()); + // the largest correlated frame the service may build under the default + // 4096-byte cap: max header (11) + payload + crc (4) <= 4096 + CHECK(sf::kMaxHeaderSize + (4096 - sf::kMaxHeaderSize - sf::kCrcSize) + sf::kCrcSize == 4096); } int main() { diff --git a/components/system/README.md b/components/system/README.md index e48678d5d8..20fe8d0374 100644 --- a/components/system/README.md +++ b/components/system/README.md @@ -63,6 +63,10 @@ little-endian. See `include/detail/system_protocol.hpp` (host-buildable, with | `0x83` OK | D→H | `[request_type u8]` | | `0x84` ERROR | D→H | `[request_type u8][code u32][utf8 message]` — code is the POSIX errno of the chosen `std::errc` (the message is authoritative) | +Every reply echoes the request frame's optional `stream_frame` correlation id, +so a host that stamps its requests can pair replies with them and drop a late +reply to a request it already timed out (the console does). + INFO tags: 1 chip model (str), 2 chip revision (u16), 3 cores (u8), 4 chip features (u32), 5 IDF version, 6 project name, 7 app version, 8 build date, 9 build time, 10 ELF SHA-256 (32 bytes), 11 running partition, 12 boot partition, diff --git a/components/system/example/CMakeLists.txt b/components/system/example/CMakeLists.txt index 28de1dafed..0f83e5f18e 100644 --- a/components/system/example/CMakeLists.txt +++ b/components/system/example/CMakeLists.txt @@ -2,7 +2,7 @@ # in this exact order for cmake to work correctly cmake_minimum_required(VERSION 3.20) -set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD 20) # This example needs the managed `espressif/esp_tinyusb` component (required by # usb_device). It supports two build modes: diff --git a/components/system/include/detail/system_protocol.hpp b/components/system/include/detail/system_protocol.hpp index 23108b63bb..717f81ac82 100644 --- a/components/system/include/detail/system_protocol.hpp +++ b/components/system/include/detail/system_protocol.hpp @@ -21,6 +21,8 @@ // 0x84 ERROR [request_type u8][code u32][utf8 message] // code = the POSIX errno value of the std::errc the service // chose (informational; the message is authoritative) +// Every reply echoes the request frame's optional correlation id, so a host +// that stamps its requests can pair replies with them and drop stale ones. // A reboot request is acknowledged with OK first; the device restarts after // the requested delay (clamped to at least Config::min_restart_delay). @@ -91,10 +93,13 @@ inline constexpr bool is_reply(Type type) { return (static_cast(type) & /// Build an encoded frame for a system message (device->host types map to the /// frame reply flag). +/// @param correlation The stream_frame correlation id to carry: a reply echoes +/// the request's, so a host can pair a reply with its request. inline std::vector build_frame(Type type, std::span payload = {}, - uint8_t module = kModule) { + uint8_t module = kModule, + std::optional correlation = std::nullopt) { return espp::stream_frame::build_frame(is_reply(type), module, static_cast(type), - payload); + payload, correlation); } // ---- INFO record encoding ------------------------------------------------------ diff --git a/components/system/include/system_service.hpp b/components/system/include/system_service.hpp index eec956fc74..afc41d2ded 100644 --- a/components/system/include/system_service.hpp +++ b/components/system/include/system_service.hpp @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -173,7 +174,7 @@ class SystemService : public BaseComponent { void handle(const espp::stream_frame::Frame &frame) { if (frame.module != module_id() || frame.is_reply()) return; - handle_frame(frame.type, frame.payload); + handle_frame(frame.type, frame.payload, frame.correlation); } /// @brief Feed received transport bytes (standalone use, without a Dispatcher). @@ -199,7 +200,10 @@ class SystemService : public BaseComponent { * sent), false if it was ignored. * @note The `send` and veto callbacks run after the internal mutex is released. */ - bool handle_frame(uint8_t type, std::span payload) { + /// @param correlation The request frame's correlation id, if it carried one; + /// every reply echoes it so the host can pair them. + bool handle_frame(uint8_t type, std::span payload, + std::optional correlation = std::nullopt) { namespace proto = espp::detail::system_protocol; std::error_code ec; switch (static_cast(type)) { @@ -209,7 +213,7 @@ class SystemService : public BaseComponent { std::lock_guard lock(mutex_); info = build_info(); } - send(proto::build_frame(Type::Info, info, module_id())); + send(proto::build_frame(Type::Info, info, module_id(), correlation)); return true; } case Type::Reboot: @@ -217,32 +221,35 @@ class SystemService : public BaseComponent { const bool bootloader = static_cast(type) == Type::RebootToBootloader; const auto delay = proto::decode_delay(payload); if (!delay) { - send_error(type, std::errc::invalid_argument, "malformed request (expected u16 delay_ms)"); + send_error(type, std::errc::invalid_argument, "malformed request (expected u16 delay_ms)", + correlation); return true; } const bool allowed = bootloader ? config_.allow_bootloader : config_.allow_reboot; if (!allowed) { send_error(type, std::errc::operation_not_permitted, bootloader ? "reboot into bootloader is disabled on this device" - : "reboot is disabled on this device"); + : "reboot is disabled on this device", + correlation); return true; } if (bootloader && !SystemControl::bootloader_reboot_supported()) { send_error(type, std::errc::operation_not_supported, - "this chip has no software path into download mode (use the BOOT strap)"); + "this chip has no software path into download mode (use the BOOT strap)", + correlation); return true; } // the veto runs outside the lock: the application may take its own locks if (config_.on_reboot_request && !config_.on_reboot_request(bootloader ? RebootKind::Bootloader : RebootKind::Reboot)) { send_error(type, std::errc::operation_canceled, - "refused by the application (try again later)"); + "refused by the application (try again later)", correlation); return true; } const auto wait = std::max(std::chrono::milliseconds(*delay), config_.min_restart_delay); // reply first, then restart from a detached thread so the reply leaves // the transport and the caller's task (the transport worker) is never blocked - send(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id())); + send(proto::build_frame(Type::Ok, proto::encode_ok(type), module_id(), correlation)); logger_.info("{} in {} ms", bootloader ? "rebooting into the bootloader" : "rebooting", wait.count()); if (bootloader) @@ -269,14 +276,15 @@ class SystemService : public BaseComponent { config_.send(frame); } - void send_error(uint8_t request_type, std::errc errc, std::string_view message) { + void send_error(uint8_t request_type, std::errc errc, std::string_view message, + std::optional correlation = std::nullopt) { namespace proto = espp::detail::system_protocol; logger_.warn("{} (type 0x{:02x})", message, request_type); send(proto::build_frame( Type::Error, proto::encode_error(request_type, static_cast(std::make_error_code(errc).value()), message), - module_id())); + module_id(), correlation)); } private: diff --git a/components/system/test/system_host_test.cpp b/components/system/test/system_host_test.cpp index 13db81cbc7..ec273db4f4 100644 --- a/components/system/test/system_host_test.cpp +++ b/components/system/test/system_host_test.cpp @@ -156,6 +156,22 @@ static void test_requests_and_replies() { const auto frames = parser.feed(stream); CHECK(frames.size() == 2 && !frames[0].is_reply() && frames[0].payload.size() == 2 && frames[1].is_reply() && frames[1].module == 11); + // correlation: a request may carry a u16 id; every reply SystemService builds + // (INFO, OK, ERROR) passes the request's id through build_frame, so it is + // echoed; a request without one gets a reply without one + const auto creq = sp::build_frame(sp::Type::GetInfo, {}, 7, 0x1234); + const auto cf = sf::StreamParser{}.feed(creq); + CHECK(cf.size() == 1 && cf[0].correlation == std::optional(0x1234) && + (creq[2] & 0x02) != 0 && creq.size() == 11 + 4); + for (const auto t : {sp::Type::Info, sp::Type::Ok, sp::Type::Error}) { + const auto crep = sp::build_frame(t, sp::encode_ok(1), 7, cf[0].correlation); + const auto cr = sf::StreamParser{}.feed(crep); + CHECK(cr.size() == 1 && cr[0].is_reply() && + cr[0].correlation == std::optional(0x1234)); + } + const auto plain = + sf::StreamParser{}.feed(sp::build_frame(sp::Type::Ok, sp::encode_ok(1), 7, std::nullopt)); + CHECK(plain.size() == 1 && !plain[0].has_correlation()); } int main() { diff --git a/components/system/web/system_console.html b/components/system/web/system_console.html index 8d40de3db0..afda30362b 100644 --- a/components/system/web/system_console.html +++ b/components/system/web/system_console.html @@ -448,21 +448,34 @@

Log

// skipped, non-frame bytes so a console+protocol CDC stream can render // its text). // =================================================================== - function buildFrame(module, type, payload) { + // `correlation` (u16, optional): stamped on every service request so the + // reply -- which echoes it -- can be paired with its request and a late + // reply to a timed-out request is recognised and dropped. It sits after + // `type` and sets flags bit1 (the header grows to 11 bytes). + function buildFrame(module, type, payload, correlation) { payload = payload || new Uint8Array(0); if (payload.length > MAX_PAYLOAD) throw new Error("payload exceeds " + MAX_PAYLOAD + " bytes"); - const frame = new Uint8Array(HEADER_SIZE + payload.length + CRC_SIZE); + const ext = correlation == null ? 0 : 2; + const headerSize = HEADER_SIZE + ext; + const frame = new Uint8Array(headerSize + payload.length + CRC_SIZE); const view = new DataView(frame.buffer); view.setUint16(0, 0x4F54, true); - frame[2] = FLAGS_REQUEST; + frame[2] = FLAGS_REQUEST | (ext ? FLAG_CORRELATION : 0); frame[3] = module; frame[4] = type; - view.setUint32(5, payload.length, true); - frame.set(payload, HEADER_SIZE); - view.setUint32(HEADER_SIZE + payload.length, - crc32(frame.subarray(0, HEADER_SIZE + payload.length)), true); + if (ext) view.setUint16(5, correlation & 0xFFFF, true); + view.setUint32(5 + ext, payload.length, true); + frame.set(payload, headerSize); + view.setUint32(headerSize + payload.length, + crc32(frame.subarray(0, headerSize + payload.length)), true); return frame; } + let correlationCounter = 0; + function nextCorrelation() { correlationCounter = (correlationCounter + 1) & 0xFFFF; return correlationCounter; } + // Whether this device has echoed a correlation id yet: once it has, a reply + // WITHOUT one can only be stale / foreign and is ignored; before that, an + // uncorrelated reply is accepted (older firmware that does not echo ids). + let correlationSeen = false; class StreamParser { constructor() { this.buf = new Uint8Array(0); this.lastActivity = 0; } @@ -688,7 +701,14 @@

Log

return null; } await port.open({ baudRate: SERIAL_BAUD }); - const writer = port.writable.getWriter(); + let writer; + try { + writer = port.writable.getWriter(); + } catch (e) { + // the port is open but unusable: do not leave it open behind us + try { await port.close(); } catch (_) {} + throw e; + } const t = { kind: "serial", name: "serial port (CDC console + protocol)", @@ -699,6 +719,10 @@

Log

async close() { this.closing = true; try { if (this.reader) await this.reader.cancel(); } catch (_) {} + // finish (or abandon) the writable stream BEFORE releasing the lock: + // a pending write would otherwise make releaseLock() throw and leave + // the port locked and open + try { await writer.close(); } catch (_) { try { await writer.abort(); } catch (_2) {} } try { writer.releaseLock(); } catch (_) {} try { await port.close(); } catch (_) {} }, @@ -797,6 +821,7 @@

Log

consoleCarry = ""; moduleReady = false; // action buttons stay disabled until discovery resolved the ids monitorPresent = false; + correlationSeen = false; // learned per device resetDeviceState(); // nothing from a previous device may carry over updateUI(); setStatus("connected", "Connected"); @@ -882,6 +907,17 @@

Log

const pending = isSys ? pendingSys : isMon ? pendingMon : null; const errType = isSys ? SYS.ERROR : MON.ERROR; if (pending && (pending.expect.has(frame.type) || frame.type === errType)) { + // pair the reply with the in-flight request by correlation id + if (frame.correlation !== null) { + if (frame.correlation !== pending.corr) { + logLine("sys", "Dropping a stale " + (names[frame.type] || "reply") + " (correlation " + frame.correlation + ", expected " + pending.corr + ")."); + return; + } + correlationSeen = true; + } else if (correlationSeen) { + logLine("sys", "Dropping an uncorrelated " + (names[frame.type] || "reply") + " (the device echoes correlation ids)."); + return; + } clearTimeout(pending.timer); if (isSys) pendingSys = null; else pendingMon = null; if (frame.type === errType) { @@ -908,10 +944,12 @@

Log

if (!moduleReady) { reject(new Error("module id not resolved yet (discovery pending)")); return; } if (which === "mon" && !monitorPresent) { reject(new Error("the device does not advertise the monitor module")); return; } const module = which === "sys" ? moduleSystem : moduleMonitor; - const frame = buildFrame(module, type, payload); - if (els.logFrames.checked) logLine("tx", (which === "sys" ? "system " : "monitor ") + names[type] + " " + hex(frame)); + const corr = nextCorrelation(); + const frame = buildFrame(module, type, payload, corr); + if (els.logFrames.checked) logLine("tx", (which === "sys" ? "system " : "monitor ") + names[type] + " #" + corr + " " + hex(frame)); const pending = { expect: new Set(expectTypes), + corr, resolve, reject, timer: setTimeout(() => { const p = which === "sys" ? pendingSys : pendingMon; @@ -1101,7 +1139,8 @@

Log

const rec = new Map(); let i = 0, unknown = 0; const dec = new TextDecoder(); - while (i + 2 <= payload.length) { + while (i < payload.length) { + if (i + 2 > payload.length) throw new Error("truncated INFO record header at byte " + i); const tag = payload[i], len = payload[i + 1]; i += 2; if (i + len > payload.length) throw new Error("truncated INFO record (tag " + tag + ")"); diff --git a/doc/en/core/monitor.rst b/doc/en/core/monitor.rst index 5fe3e02632..60a26aa43c 100644 --- a/doc/en/core/monitor.rst +++ b/doc/en/core/monitor.rst @@ -77,11 +77,13 @@ an instance and hosts find it through discovery by its protocol id). have are left out), ``GET_TASKS`` with the ``TaskMonitor`` table (name, CPU %, stack high-water mark, priority, core — it needs ``CONFIG_FREERTOS_USE_TRACE_FACILITY`` and -``CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS``, else the list is empty; the reply -is capped so the whole frame fits ``Config::max_frame_bytes``, 4096 by -default, tasks beyond it being dropped from the end), and +``CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS``, else the list is empty; both +replies are capped so the whole frame fits ``Config::max_frame_bytes``, 4096 +by default, regions / tasks beyond it being dropped from the end), and ``SET_STREAM`` starts a task that sends either or both periodically so a host -can plot them live. The wire codec (``detail/monitor_protocol.hpp``) is +can plot them live. Replies echo the request frame's correlation id so a host +can pair them and drop stale ones (streamed events carry none). The wire codec +(``detail/monitor_protocol.hpp``) is host-buildable and unit-tested (``test/monitor_host_test.cpp``). The hosted `espp System Console `_ web app renders the heap gauges and a live task table; see the diff --git a/doc/en/system/system.rst b/doc/en/system/system.rst index 5dc843b045..8f5d730e78 100644 --- a/doc/en/system/system.rst +++ b/doc/en/system/system.rst @@ -41,7 +41,9 @@ The `SystemService` class serves both over **any byte stream** as a module id 7 by default; ``Config::module`` moves an instance and hosts find it through discovery by its protocol id). ``GET_INFO`` answers with a list of tagged records (``[tag u8][len u8][value]``) a host decodes while skipping -tags it does not know, so fields can be added without a version bump. +tags it does not know, so fields can be added without a version bump. Every +reply echoes the request frame's correlation id, so a host that stamps its +requests can pair replies with them and drop stale ones. ``REBOOT`` and ``REBOOT_TO_BOOTLOADER`` reply ``OK`` first and restart after the requested delay (clamped to ``Config::min_restart_delay``). Both are guarded: ``Config::allow_reboot`` / ``allow_bootloader`` switch them off, the From ef55e37c4ec3b9fe3144da5eddd09692b4732364 Mon Sep 17 00:00:00 2001 From: William Emfinger Date: Wed, 30 Sep 2026 21:42:21 -0400 Subject: [PATCH 7/7] docs(system,monitor): console screenshots in the READMEs and doc pages Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU --- components/monitor/README.md | 4 +++- components/system/README.md | 9 +++++++++ doc/en/core/monitor.rst | 5 +++++ doc/en/system/system.rst | 10 ++++++++++ 4 files changed, 27 insertions(+), 1 deletion(-) diff --git a/components/monitor/README.md b/components/monitor/README.md index 0a84ce93d4..355482b41f 100644 --- a/components/monitor/README.md +++ b/components/monitor/README.md @@ -41,7 +41,9 @@ events carry none. The wire codec lives in [system console](https://esp-cpp.github.io/espp/apps/system_console.html) web app renders heap gauges and a live task table from it; the [system](../system) component's example exposes it over USB together with -`espp::SystemService`. +`espp::SystemService`: + +espp System Console streaming the task table from MonitorService (name, CPU %, stack high-water mark, priority, core) ## Example diff --git a/components/system/README.md b/components/system/README.md index 20fe8d0374..54bfc72d0b 100644 --- a/components/system/README.md +++ b/components/system/README.md @@ -38,6 +38,15 @@ web app (`web/system_console.html`) talks to the service over WebUSB or Web Serial, and to the [monitor](../monitor) component's `MonitorService` (heap regions + task table, live) when the device advertises it. +The console connected to the example over WebUSB: device info, the two restart +buttons and the heap gauges + +espp System Console: device info, reboot / bootloader controls and heap gauges + +and its live task table, streamed by `MonitorService` + +espp System Console streaming the task table (name, CPU %, stack high-water mark, priority, core) + **Table of Contents** diff --git a/doc/en/core/monitor.rst b/doc/en/core/monitor.rst index 60a26aa43c..ed8786b26a 100644 --- a/doc/en/core/monitor.rst +++ b/doc/en/core/monitor.rst @@ -90,6 +90,11 @@ web app renders the heap gauges and a live task table; see the :doc:`system <../system/system>` component's example, which exposes both services over USB. +.. image:: https://github.com/user-attachments/assets/c2c962bf-debb-44c7-96a7-95527218d953 + :alt: espp System Console streaming the task table from MonitorService (name, CPU %, stack high-water mark, priority, core) + :width: 100% + :target: https://esp-cpp.github.io/espp/apps/system_console.html + Monitor Service API Reference ----------------------------- diff --git a/doc/en/system/system.rst b/doc/en/system/system.rst index 8f5d730e78..c8dfcee802 100644 --- a/doc/en/system/system.rst +++ b/doc/en/system/system.rst @@ -60,6 +60,16 @@ it doubles as a serial monitor): a device-info panel, the two restart buttons :doc:`monitor <../core/monitor>` component's ``MonitorService`` — heap-region gauges and a live, sortable task table with a stream toggle. +.. image:: https://github.com/user-attachments/assets/bc5fa520-f3a0-40de-9b53-889a887c0322 + :alt: espp System Console: device info, reboot / bootloader controls and heap gauges + :width: 100% + :target: https://esp-cpp.github.io/espp/apps/system_console.html + +.. image:: https://github.com/user-attachments/assets/c2c962bf-debb-44c7-96a7-95527218d953 + :alt: espp System Console streaming the task table (name, CPU %, stack high-water mark, priority, core) + :width: 100% + :target: https://esp-cpp.github.io/espp/apps/system_console.html + .. ------------------------------- Example ------------------------------------- .. toctree::